AI Cho Mọi Người → Vectorization

AI Cho Mọi Người

Vectorization

Trong bài này, chúng ta sẽ cài đặt bài toán dự đoán giá nhà dùng linear regression bằng hai cách: cài đặt dùng chỉ mục index và dùng những hàm trong numpy đã hỗ trợ kỹ thuật vectorization. Sau đó, chúng ta sẽ quan sát tốc độ của hai loại cài đặt.

1. Dự đoán giá nhà theo diện tích

Chúng ta sẽ sử dụng dữ liệu dự đoán giá nhà theo diện tích nhà, được hiển thị ở bảng sau

3.87	5.38
2.29	3.45
5.17	6.94
4.93	7.1
4	5.47
3.97	5.31
6.77	8.89
2.48	4.08
2.85	4.59
6.43	8.52
6.99	8.63
2.74	4.29
2.07	3.34
3.64	5.48
5.66	7.4
1.85	3.3
4.73	6.19
4.55	6.46
6.4	8.65
2.14	3.35

Code để đọc và xử lý data

import numpy as np
from numpy import genfromtxt
import matplotlib.pyplot as plt

data = genfromtxt('my_house_price_prediction.csv', delimiter=',')

m = data.shape[0]
n = data.shape[1]

x = data[:,0]
y = data[:,1]
X = np.c_[np.ones((m, 1)), x]

plt.scatter(x, y)
plt.xlabel('Diện tích nhà (x 100$m^2$)')
plt.ylabel('Giá nhà (chục lượng vàng)')
plt.show()

Cài theo cách dùng chỉ mục index

# No Vectorization - batch gradien descent

theta = np.random.randn(n)

# loss function
def loss_function(theta = theta, x=X, y=y, m=m, n=n):
    loss = 0
    for i in range(m):
        hypo_i = 0
        for j in range(n):
            hypo_i += theta[j]*X[i,j]
        loss_i = (hypo_i - y[i])**2
        loss += loss_i
    loss = (1/m)*loss
    
    return loss


# training
learning_rate = 0.01
theta = np.ones(n)
loss_list = []

epoches = 50
for itr in range(epoches):    
    dev_list = []
    for k in range(n):
        dev_sum = 0        
        
        for i in range(m):
            ## Feed forward
            hypo_i = 0
            for j in range(n):
                hypo_i += theta[j]*X[i,j]
            
            # derivative
            dev_i = (hypo_i - y[i])*X[i,k]
            dev_sum += dev_i
        dev_sum = (2/m)*dev_sum

        dev_list.append(dev_sum)
    
    theta = theta - learning_rate*np.array(dev_list)    
    loss_val = loss_function(theta)    
    loss_list.append(loss_val)
    

plt.plot(np.arange(0, epoches),loss_list)
plt.xlabel('epoch')
plt.ylabel('Giá trị loss')

Giá trị loss qua các vòng lặp

Cài theo phương pháp vectorization

# Vectorization

# Initialize theta
theta = np.random.randn(n)

def cost(theta, X=X, y=y, m=m):
    cost = np.dot(np.dot(X,theta) - y, np.dot(X,theta) - y)
    cost = (1/m)*cost
    return cost

# learning rate
learning_rate = 0.01
cost_list = []

epoches = 500
for i in range(epoches):
    output = np.dot(X,theta)
    loss_grd = output - y
    gradients = (2/m)*np.dot(np.transpose(X), loss_grd)
    
    theta = theta - learning_rate*gradients      
    cost_val = cost(theta)
    cost_list.append(cost_val)

plt.plot(np.arange(0, epoches),cost_list)
plt.xlabel('epoch')
plt.ylabel('Giá trị loss')

Giá trị loss qua các vòng lặp

2. Dự đoán giá nhà Boston

Dữ liệu nhà Boston được mô tả ở bảng sau

crim	zn	indus	chas	nox	rm	age	dis	rad	tax	ptratio	black	lstat	medv
0.00632	18	2.31	0	0.538	6.575	65.2	4.09	1	296	15.3	396.9	4.98	24
0.02731	0	7.07	0	0.469	6.421	78.9	4.9671	2	242	17.8	396.9	9.14	21.6
0.03237	0	2.18	0	0.458	6.998	45.8	6.0622	3	222	18.7	394.63	2.94	33.4
0.06905	0	2.18	0	0.458	7.147	54.2	6.0622	3	222	18.7	396.9	5.33	36.2
0.08829	12.5	7.87	0	0.524	6.012	66.6	5.5605	5	311	15.2	395.6	12.43	22.9
0.22489	12.5	7.87	0	0.524	6.377	94.3	6.3467	5	311	15.2	392.52	20.45	15
0.11747	12.5	7.87	0	0.524	6.009	82.9	6.2267	5	311	15.2	396.9	13.27	18.9
0.09378	12.5	7.87	0	0.524	5.889	39	5.4509	5	311	15.2	390.5	15.71	21.7
0.62976	0	8.14	0	0.538	5.949	61.8	4.7075	4	307	21	396.9	8.26	20.4
0.63796	0	8.14	0	0.538	6.096	84.5	4.4619	4	307	21	380.02	10.26	18.2
0.62739	0	8.14	0	0.538	5.834	56.5	4.4986	4	307	21	395.62	8.47	19.9
1.05393	0	8.14	0	0.538	5.935	29.3	4.4986	4	307	21	386.85	6.58	23.1
0.80271	0	8.14	0	0.538	5.456	36.6	3.7965	4	307	21	288.99	11.69	20.2
1.25179	0	8.14	0	0.538	5.57	98.1	3.7979	4	307	21	376.57	21.02	13.6
0.85204	0	8.14	0	0.538	5.965	89.2	4.0123	4	307	21	392.53	13.83	19.6
1.23247	0	8.14	0	0.538	6.142	91.7	3.9769	4	307	21	396.9	18.72	15.2
0.98843	0	8.14	0	0.538	5.813	100	4.0952	4	307	21	394.54	19.88	14.5
0.95577	0	8.14	0	0.538	6.047	88.8	4.4534	4	307	21	306.38	17.28	14.8
1.13081	0	8.14	0	0.538	5.713	94.1	4.233	4	307	21	360.17	22.6	12.7
1.35472	0	8.14	0	0.538	6.072	100	4.175	4	307	21	376.73	13.04	14.5
1.61282	0	8.14	0	0.538	6.096	96.9	3.7598	4	307	21	248.31	20.34	13.5
0.17505	0	5.96	0	0.499	5.966	30.2	3.8473	5	279	19.2	393.43	10.13	24.7
0.02763	75	2.95	0	0.428	6.595	21.8	5.4011	3	252	18.3	395.63	4.32	30.8
0.03359	75	2.95	0	0.428	7.024	15.8	5.4011	3	252	18.3	395.62	1.98	34.9
0.1415	0	6.91	0	0.448	6.169	6.6	5.7209	3	233	17.9	383.37	5.81	25.3
0.15936	0	6.91	0	0.448	6.211	6.5	5.7209	3	233	17.9	394.46	7.44	24.7
0.12269	0	6.91	0	0.448	6.069	40	5.7209	3	233	17.9	389.39	9.55	21.2
0.17142	0	6.91	0	0.448	5.682	33.8	5.1004	3	233	17.9	396.9	10.21	19.3
0.18836	0	6.91	0	0.448	5.786	33.3	5.1004	3	233	17.9	396.9	14.15	20
0.22927	0	6.91	0	0.448	6.03	85.5	5.6894	3	233	17.9	392.74	18.8	16.6
0.21977	0	6.91	0	0.448	5.602	62	6.0877	3	233	17.9	396.9	16.2	19.4
0.08873	21	5.64	0	0.439	5.963	45.7	6.8147	4	243	16.8	395.56	13.45	19.7
0.04337	21	5.64	0	0.439	6.115	63	6.8147	4	243	16.8	393.97	9.43	20.5
0.04981	21	5.64	0	0.439	5.998	21.4	6.8147	4	243	16.8	396.9	8.43	23.4
0.0136	75	4	0	0.41	5.888	47.6	7.3197	3	469	21.1	396.9	14.8	18.9
0.01311	90	1.22	0	0.403	7.249	21.9	8.6966	5	226	17.9	395.93	4.81	35.4
0.02055	85	0.74	0	0.41	6.383	35.7	9.1876	2	313	17.3	396.9	5.77	24.7
0.01432	100	1.32	0	0.411	6.816	40.5	8.3248	5	256	15.1	392.9	3.95	31.6
0.15445	25	5.13	0	0.453	6.145	29.2	7.8148	8	284	19.7	390.68	6.86	23.3
0.14932	25	5.13	0	0.453	5.741	66.2	7.2254	8	284	19.7	395.11	13.15	18.7
0.17171	25	5.13	0	0.453	5.966	93.4	6.8185	8	284	19.7	378.08	14.44	16
0.1265	25	5.13	0	0.453	6.762	43.4	7.9809	8	284	19.7	395.58	9.5	25
0.01951	17.5	1.38	0	0.4161	7.104	59.5	9.2229	3	216	18.6	393.24	8.05	33
0.03584	80	3.37	0	0.398	6.29	17.8	6.6115	4	337	16.1	396.9	4.67	23.5
0.04379	80	3.37	0	0.398	5.787	31.1	6.6115	4	337	16.1	396.9	10.24	19.4
0.05789	12.5	6.07	0	0.409	5.878	21.4	6.498	4	345	18.9	396.21	8.1	22
0.13554	12.5	6.07	0	0.409	5.594	36.8	6.498	4	345	18.9	396.9	13.09	17.4
0.08826	0	10.81	0	0.413	6.417	6.6	5.2873	4	305	19.2	383.73	6.72	24.2
0.09164	0	10.81	0	0.413	6.065	7.8	5.2873	4	305	19.2	390.91	5.52	22.8
0.19539	0	10.81	0	0.413	6.245	6.2	5.2873	4	305	19.2	377.17	7.54	23.4
0.07896	0	12.83	0	0.437	6.273	6	4.2515	5	398	18.7	394.92	6.78	24.1
0.09512	0	12.83	0	0.437	6.286	45	4.5026	5	398	18.7	383.23	8.94	21.4
0.10153	0	12.83	0	0.437	6.279	74.5	4.0522	5	398	18.7	373.66	11.97	20
0.08707	0	12.83	0	0.437	6.14	45.8	4.0905	5	398	18.7	386.96	10.27	20.8
0.04113	25	4.86	0	0.426	6.727	33.5	5.4007	4	281	19	396.9	5.29	28
0.04462	25	4.86	0	0.426	6.619	70.4	5.4007	4	281	19	395.63	7.22	23.9
0.03551	25	4.86	0	0.426	6.167	46.7	5.4007	4	281	19	390.64	7.51	22.9
0.05059	0	4.49	0	0.449	6.389	48	4.7794	3	247	18.5	396.9	9.62	23.9
0.05735	0	4.49	0	0.449	6.63	56.1	4.4377	3	247	18.5	392.3	6.53	26.6
0.05188	0	4.49	0	0.449	6.015	45.1	4.4272	3	247	18.5	395.99	12.86	22.5
0.07151	0	4.49	0	0.449	6.121	56.8	3.7476	3	247	18.5	395.15	8.44	22.2
0.0566	0	3.41	0	0.489	7.007	86.3	3.4217	2	270	17.8	396.9	5.5	23.6
0.05302	0	3.41	0	0.489	7.079	63.1	3.4145	2	270	17.8	396.06	5.7	28.7
0.04684	0	3.41	0	0.489	6.417	66.1	3.0923	2	270	17.8	392.18	8.81	22.6
0.02875	28	15.04	0	0.464	6.211	28.9	3.6659	4	270	18.2	396.33	6.21	25
0.04294	28	15.04	0	0.464	6.249	77.3	3.615	4	270	18.2	396.9	10.59	20.6
0.11504	0	2.89	0	0.445	6.163	69.6	3.4952	2	276	18	391.83	11.34	21.4
0.14866	0	8.56	0	0.52	6.727	79.9	2.7778	5	384	20.9	394.76	9.42	27.5
0.11432	0	8.56	0	0.52	6.781	71.3	2.8561	5	384	20.9	395.58	7.67	26.5
0.22876	0	8.56	0	0.52	6.405	85.4	2.7147	5	384	20.9	70.8	10.63	18.6
0.21161	0	8.56	0	0.52	6.137	87.4	2.7147	5	384	20.9	394.47	13.44	19.3
0.1712	0	8.56	0	0.52	5.836	91.9	2.211	5	384	20.9	395.67	18.66	19.5
0.13117	0	8.56	0	0.52	6.127	85.2	2.1224	5	384	20.9	387.69	14.09	20.4
0.12802	0	8.56	0	0.52	6.474	97.1	2.4329	5	384	20.9	395.24	12.27	19.8
0.26363	0	8.56	0	0.52	6.229	91.2	2.5451	5	384	20.9	391.23	15.55	19.4
0.10084	0	10.01	0	0.547	6.715	81.6	2.6775	6	432	17.8	395.59	10.16	22.8
0.14231	0	10.01	0	0.547	6.254	84.2	2.2565	6	432	17.8	388.74	10.45	18.5
0.13158	0	10.01	0	0.547	6.176	72.5	2.7301	6	432	17.8	393.3	12.04	21.2
0.15098	0	10.01	0	0.547	6.021	82.6	2.7474	6	432	17.8	394.51	10.3	19.2
0.13058	0	10.01	0	0.547	5.872	73.1	2.4775	6	432	17.8	338.63	15.37	20.4
0.14476	0	10.01	0	0.547	5.731	65.2	2.7592	6	432	17.8	391.5	13.61	19.3
0.06899	0	25.65	0	0.581	5.87	69.7	2.2577	2	188	19.1	389.15	14.37	22
0.07165	0	25.65	0	0.581	6.004	84.1	2.1974	2	188	19.1	377.67	14.27	20.3
0.09299	0	25.65	0	0.581	5.961	92.9	2.0869	2	188	19.1	378.09	17.93	20.5
0.15038	0	25.65	0	0.581	5.856	97	1.9444	2	188	19.1	370.31	25.41	17.3
0.09849	0	25.65	0	0.581	5.879	95.8	2.0063	2	188	19.1	379.38	17.58	18.8
0.38735	0	25.65	0	0.581	5.613	95.6	1.7572	2	188	19.1	359.29	27.26	15.7
0.25915	0	21.89	0	0.624	5.693	96	1.7883	4	437	21.2	392.11	17.19	16.2
0.32543	0	21.89	0	0.624	6.431	98.8	1.8125	4	437	21.2	396.9	15.39	18
1.19294	0	21.89	0	0.624	6.326	97.7	2.271	4	437	21.2	396.9	12.26	19.6
0.32982	0	21.89	0	0.624	5.822	95.4	2.4699	4	437	21.2	388.69	15.03	18.4
0.97617	0	21.89	0	0.624	5.757	98.4	2.346	4	437	21.2	262.76	17.31	15.6
0.32264	0	21.89	0	0.624	5.942	93.5	1.9669	4	437	21.2	378.25	16.9	17.4
0.35233	0	21.89	0	0.624	6.454	98.4	1.8498	4	437	21.2	394.08	14.59	17.1
0.2498	0	21.89	0	0.624	5.857	98.2	1.6686	4	437	21.2	392.04	21.32	13.3
0.54452	0	21.89	0	0.624	6.151	97.9	1.6687	4	437	21.2	396.9	18.46	17.8
1.62864	0	21.89	0	0.624	5.019	100	1.4394	4	437	21.2	396.9	34.41	14.4
3.32105	0	19.58	1	0.871	5.403	100	1.3216	5	403	14.7	396.9	26.82	13.4
2.37934	0	19.58	0	0.871	6.13	100	1.4191	5	403	14.7	172.91	27.8	13.8
2.36862	0	19.58	0	0.871	4.926	95.7	1.4608	5	403	14.7	391.71	29.53	14.6
2.33099	0	19.58	0	0.871	5.186	93.8	1.5296	5	403	14.7	356.99	28.32	17.8
2.73397	0	19.58	0	0.871	5.597	94.9	1.5257	5	403	14.7	351.85	21.45	15.4
1.6566	0	19.58	0	0.871	6.122	97.3	1.618	5	403	14.7	372.8	14.1	21.5
2.14918	0	19.58	0	0.871	5.709	98.5	1.6232	5	403	14.7	261.95	15.79	19.4
1.41385	0	19.58	1	0.871	6.129	96	1.7494	5	403	14.7	321.02	15.12	17
2.44668	0	19.58	0	0.871	5.272	94	1.7364	5	403	14.7	88.63	16.14	13.1
1.34284	0	19.58	0	0.605	6.066	100	1.7573	5	403	14.7	353.89	6.43	24.3
1.42502	0	19.58	0	0.871	6.51	100	1.7659	5	403	14.7	364.31	7.39	23.3
1.27346	0	19.58	1	0.605	6.25	92.6	1.7984	5	403	14.7	338.92	5.5	27
1.46336	0	19.58	0	0.605	7.489	90.8	1.9709	5	403	14.7	374.43	1.73	50
1.51902	0	19.58	1	0.605	8.375	93.9	2.162	5	403	14.7	388.45	3.32	50
2.24236	0	19.58	0	0.605	5.854	91.8	2.422	5	403	14.7	395.11	11.64	22.7
2.924	0	19.58	0	0.605	6.101	93	2.2834	5	403	14.7	240.16	9.81	25
2.01019	0	19.58	0	0.605	7.929	96.2	2.0459	5	403	14.7	369.3	3.7	50
1.80028	0	19.58	0	0.605	5.877	79.2	2.4259	5	403	14.7	227.61	12.14	23.8
2.44953	0	19.58	0	0.605	6.402	95.2	2.2625	5	403	14.7	330.04	11.32	22.3
1.20742	0	19.58	0	0.605	5.875	94.6	2.4259	5	403	14.7	292.29	14.43	17.4
2.3139	0	19.58	0	0.605	5.88	97.3	2.3887	5	403	14.7	348.13	12.03	19.1
0.13914	0	4.05	0	0.51	5.572	88.5	2.5961	5	296	16.6	396.9	14.69	23.1
0.09178	0	4.05	0	0.51	6.416	84.1	2.6463	5	296	16.6	395.5	9.04	23.6
0.08447	0	4.05	0	0.51	5.859	68.7	2.7019	5	296	16.6	393.23	9.64	22.6
0.06664	0	4.05	0	0.51	6.546	33.1	3.1323	5	296	16.6	390.96	5.33	29.4
0.07022	0	4.05	0	0.51	6.02	47.2	3.5549	5	296	16.6	393.23	10.11	23.2
0.05425	0	4.05	0	0.51	6.315	73.4	3.3175	5	296	16.6	395.6	6.29	24.6
0.06642	0	4.05	0	0.51	6.86	74.4	2.9153	5	296	16.6	391.27	6.92	29.9
0.0578	0	2.46	0	0.488	6.98	58.4	2.829	3	193	17.8	396.9	5.04	37.2
0.06588	0	2.46	0	0.488	7.765	83.3	2.741	3	193	17.8	395.56	7.56	39.8
0.06888	0	2.46	0	0.488	6.144	62.2	2.5979	3	193	17.8	396.9	9.45	36.2
0.09103	0	2.46	0	0.488	7.155	92.2	2.7006	3	193	17.8	394.12	4.82	37.9
0.10008	0	2.46	0	0.488	6.563	95.6	2.847	3	193	17.8	396.9	5.68	32.5
0.05602	0	2.46	0	0.488	7.831	53.6	3.1992	3	193	17.8	392.63	4.45	50
0.07875	45	3.44	0	0.437	6.782	41.1	3.7886	5	398	15.2	393.87	6.68	32
0.0837	45	3.44	0	0.437	7.185	38.9	4.5667	5	398	15.2	396.9	5.39	34.9
0.09068	45	3.44	0	0.437	6.951	21.5	6.4798	5	398	15.2	377.68	5.1	37
0.06911	45	3.44	0	0.437	6.739	30.8	6.4798	5	398	15.2	389.71	4.69	30.5
0.08664	45	3.44	0	0.437	7.178	26.3	6.4798	5	398	15.2	390.49	2.87	36.4
0.02187	60	2.93	0	0.401	6.8	9.9	6.2196	1	265	15.6	393.37	5.03	31.1
0.01439	60	2.93	0	0.401	6.604	18.8	6.2196	1	265	15.6	376.7	4.38	29.1
0.04666	80	1.52	0	0.404	7.107	36.6	7.309	2	329	12.6	354.31	8.61	30.3
0.01778	95	1.47	0	0.403	7.135	13.9	7.6534	3	402	17	384.3	4.45	32.9
0.03445	82.5	2.03	0	0.415	6.162	38.4	6.27	2	348	14.7	393.77	7.43	24.1
0.0351	95	2.68	0	0.4161	7.853	33.2	5.118	4	224	14.7	392.78	3.81	48.5
0.02009	95	2.68	0	0.4161	8.034	31.9	5.118	4	224	14.7	390.55	2.88	50
0.13642	0	10.59	0	0.489	5.891	22.3	3.9454	4	277	18.6	396.9	10.87	22.6
0.22969	0	10.59	0	0.489	6.326	52.5	4.3549	4	277	18.6	394.87	10.97	24.4
0.13587	0	10.59	1	0.489	6.064	59.1	4.2392	4	277	18.6	381.32	14.66	24.4
0.37578	0	10.59	1	0.489	5.404	88.6	3.665	4	277	18.6	395.24	23.98	19.3
0.14052	0	10.59	0	0.489	6.375	32.3	3.9454	4	277	18.6	385.81	9.38	28.1
0.28955	0	10.59	0	0.489	5.412	9.8	3.5875	4	277	18.6	348.93	29.55	23.7
0.0456	0	13.89	1	0.55	5.888	56	3.1121	5	276	16.4	392.8	13.51	23.3
0.40771	0	6.2	1	0.507	6.164	91.3	3.048	8	307	17.4	395.24	21.46	21.7
0.62356	0	6.2	1	0.507	6.879	77.7	3.2721	8	307	17.4	390.39	9.93	27.5
0.6147	0	6.2	0	0.507	6.618	80.8	3.2721	8	307	17.4	396.9	7.6	30.1
0.31533	0	6.2	0	0.504	8.266	78.3	2.8944	8	307	17.4	385.05	4.14	44.8
0.52693	0	6.2	0	0.504	8.725	83	2.8944	8	307	17.4	382	4.63	50
0.38214	0	6.2	0	0.504	8.04	86.5	3.2157	8	307	17.4	387.38	3.13	37.6
0.41238	0	6.2	0	0.504	7.163	79.9	3.2157	8	307	17.4	372.08	6.36	31.6
0.44178	0	6.2	0	0.504	6.552	21.4	3.3751	8	307	17.4	380.34	3.76	31.5
0.537	0	6.2	0	0.504	5.981	68.1	3.6715	8	307	17.4	378.35	11.65	24.3
0.57529	0	6.2	0	0.507	8.337	73.3	3.8384	8	307	17.4	385.91	2.47	41.7
0.33147	0	6.2	0	0.507	8.247	70.4	3.6519	8	307	17.4	378.95	3.95	48.3
0.44791	0	6.2	1	0.507	6.726	66.5	3.6519	8	307	17.4	360.2	8.05	29
0.33045	0	6.2	0	0.507	6.086	61.5	3.6519	8	307	17.4	376.75	10.88	24
0.52058	0	6.2	1	0.507	6.631	76.5	4.148	8	307	17.4	388.45	9.54	25.1
0.11329	30	4.93	0	0.428	6.897	54.3	6.3361	6	300	16.6	391.25	11.38	22
0.1029	30	4.93	0	0.428	6.358	52.9	7.0355	6	300	16.6	372.75	11.22	22.2
0.12757	30	4.93	0	0.428	6.393	7.8	7.0355	6	300	16.6	374.71	5.19	23.7
0.20608	22	5.86	0	0.431	5.593	76.5	7.9549	7	330	19.1	372.49	12.5	17.6
0.33983	22	5.86	0	0.431	6.108	34.9	8.0555	7	330	19.1	390.18	9.16	24.3
0.16439	22	5.86	0	0.431	6.433	49.1	7.8265	7	330	19.1	374.71	9.52	24.5
0.19073	22	5.86	0	0.431	6.718	17.5	7.8265	7	330	19.1	393.74	6.56	26.2
0.1403	22	5.86	0	0.431	6.487	13	7.3967	7	330	19.1	396.28	5.9	24.4
0.21409	22	5.86	0	0.431	6.438	8.9	7.3967	7	330	19.1	377.07	3.59	24.8
0.36894	22	5.86	0	0.431	8.259	8.4	8.9067	7	330	19.1	396.9	3.54	42.8
0.54011	20	3.97	0	0.647	7.203	81.8	2.1121	5	264	13	392.8	9.59	33.8
0.53412	20	3.97	0	0.647	7.52	89.4	2.1398	5	264	13	388.37	7.26	43.1
0.52014	20	3.97	0	0.647	8.398	91.5	2.2885	5	264	13	386.86	5.91	48.8
0.82526	20	3.97	0	0.647	7.327	94.5	2.0788	5	264	13	393.42	11.25	31
0.55007	20	3.97	0	0.647	7.206	91.6	1.9301	5	264	13	387.89	8.1	36.5
0.76162	20	3.97	0	0.647	5.56	62.8	1.9865	5	264	13	392.4	10.45	22.8
0.7857	20	3.97	0	0.647	7.014	84.6	2.1329	5	264	13	384.07	14.79	30.7
0.5405	20	3.97	0	0.575	7.47	52.6	2.872	5	264	13	390.3	3.16	43.5
0.16211	20	6.96	0	0.464	6.24	16.3	4.429	3	223	18.6	396.9	6.59	25.2
0.1146	20	6.96	0	0.464	6.538	58.7	3.9175	3	223	18.6	394.96	7.73	24.4
0.22188	20	6.96	1	0.464	7.691	51.8	4.3665	3	223	18.6	390.77	6.58	35.2
0.05644	40	6.41	1	0.447	6.758	32.9	4.0776	4	254	17.6	396.9	3.53	32.4
0.21038	20	3.33	0	0.4429	6.812	32.2	4.1007	5	216	14.9	396.9	4.85	35.1
0.03705	20	3.33	0	0.4429	6.968	37.2	5.2447	5	216	14.9	392.23	4.59	35.4
0.06129	20	3.33	1	0.4429	7.645	49.7	5.2119	5	216	14.9	377.07	3.01	46
0.01501	90	1.21	1	0.401	7.923	24.8	5.885	1	198	13.6	395.52	3.16	50
0.00906	90	2.97	0	0.4	7.088	20.8	7.3073	1	285	15.3	394.72	7.85	32.2
0.01096	55	2.25	0	0.389	6.453	31.9	7.3073	1	300	15.3	394.72	8.23	22
0.01965	80	1.76	0	0.385	6.23	31.5	9.0892	1	241	18.2	341.6	12.93	20.1
0.0459	52.5	5.32	0	0.405	6.315	45.6	7.3172	6	293	16.6	396.9	7.6	22.3
0.03502	80	4.95	0	0.411	6.861	27.9	5.1167	4	245	19.2	396.9	3.33	28.5
0.03615	80	4.95	0	0.411	6.63	23.4	5.1167	4	245	19.2	396.9	4.7	27.9
0.08265	0	13.92	0	0.437	6.127	18.4	5.5027	4	289	16	396.9	8.58	23.9
0.05372	0	13.92	0	0.437	6.549	51	5.9604	4	289	16	392.85	7.39	27.1
0.14103	0	13.92	0	0.437	5.79	58	6.32	4	289	16	396.9	15.84	20.3
0.03537	34	6.09	0	0.433	6.59	40.4	5.4917	7	329	16.1	395.75	9.5	22
0.09266	34	6.09	0	0.433	6.495	18.4	5.4917	7	329	16.1	383.61	8.67	26.4
0.1	34	6.09	0	0.433	6.982	17.7	5.4917	7	329	16.1	390.43	4.86	33.1
0.05515	33	2.18	0	0.472	7.236	41.1	4.022	7	222	18.4	393.68	6.93	36.1
0.05479	33	2.18	0	0.472	6.616	58.1	3.37	7	222	18.4	393.36	8.93	28.4
0.07503	33	2.18	0	0.472	7.42	71.9	3.0992	7	222	18.4	396.9	6.47	33.4
0.49298	0	9.9	0	0.544	6.635	82.5	3.3175	4	304	18.4	396.9	4.54	22.8
0.3494	0	9.9	0	0.544	5.972	76.7	3.1025	4	304	18.4	396.24	9.97	20.3
2.63548	0	9.9	0	0.544	4.973	37.8	2.5194	4	304	18.4	350.45	12.64	16.1
0.79041	0	9.9	0	0.544	6.122	52.8	2.6403	4	304	18.4	396.9	5.98	22.1
0.26169	0	9.9	0	0.544	6.023	90.4	2.834	4	304	18.4	396.3	11.72	19.4
0.25356	0	9.9	0	0.544	5.705	77.7	3.945	4	304	18.4	396.42	11.5	16.2
0.31827	0	9.9	0	0.544	5.914	83.2	3.9986	4	304	18.4	390.7	18.33	17.8
0.24522	0	9.9	0	0.544	5.782	71.7	4.0317	4	304	18.4	396.9	15.94	19.8
0.40202	0	9.9	0	0.544	6.382	67.2	3.5325	4	304	18.4	395.21	10.36	23.1
0.1676	0	7.38	0	0.493	6.426	52.3	4.5404	5	287	19.6	396.9	7.2	23.8
0.34109	0	7.38	0	0.493	6.415	40.1	4.7211	5	287	19.6	396.9	6.12	25
0.19186	0	7.38	0	0.493	6.431	14.7	5.4159	5	287	19.6	393.68	5.08	24.6
0.24103	0	7.38	0	0.493	6.083	43.7	5.4159	5	287	19.6	396.9	12.79	22.2
0.06617	0	3.24	0	0.46	5.868	25.8	5.2146	4	430	16.9	382.44	9.97	19.3
0.04544	0	3.24	0	0.46	6.144	32.2	5.8736	4	430	16.9	368.57	9.09	19.8
0.05083	0	5.19	0	0.515	6.316	38.1	6.4584	5	224	20.2	389.71	5.68	22.2
0.03738	0	5.19	0	0.515	6.31	38.5	6.4584	5	224	20.2	389.4	6.75	20.7
0.03427	0	5.19	0	0.515	5.869	46.3	5.2311	5	224	20.2	396.9	9.8	19.5
0.03306	0	5.19	0	0.515	6.059	37.3	4.8122	5	224	20.2	396.14	8.51	20.6
0.05497	0	5.19	0	0.515	5.985	45.4	4.8122	5	224	20.2	396.9	9.74	19
0.06151	0	5.19	0	0.515	5.968	58.5	4.8122	5	224	20.2	396.9	9.29	18.7
0.01301	35	1.52	0	0.442	7.241	49.3	7.0379	1	284	15.5	394.74	5.49	32.7
0.02498	0	1.89	0	0.518	6.54	59.7	6.2669	1	422	15.9	389.96	8.65	16.5
0.02543	55	3.78	0	0.484	6.696	56.4	5.7321	5	370	17.6	396.9	7.18	23.9
0.03049	55	3.78	0	0.484	6.874	28.1	6.4654	5	370	17.6	387.97	4.61	31.2
0.0187	85	4.15	0	0.429	6.516	27.7	8.5353	4	351	17.9	392.43	6.36	23.1
0.01501	80	2.01	0	0.435	6.635	29.7	8.344	4	280	17	390.94	5.99	24.5
0.02899	40	1.25	0	0.429	6.939	34.5	8.7921	1	335	19.7	389.85	5.89	26.6
0.07244	60	1.69	0	0.411	5.884	18.5	10.7103	4	411	18.3	392.33	7.79	18.6
8.98296	0	18.1	1	0.77	6.212	97.4	2.1222	24	666	20.2	377.73	17.6	17.8
3.8497	0	18.1	1	0.77	6.395	91	2.5052	24	666	20.2	391.34	13.27	21.7
5.20177	0	18.1	1	0.77	6.127	83.4	2.7227	24	666	20.2	395.43	11.48	22.7
4.26131	0	18.1	0	0.77	6.112	81.3	2.5091	24	666	20.2	390.74	12.67	22.6
4.54192	0	18.1	0	0.77	6.398	88	2.5182	24	666	20.2	374.56	7.79	25
3.67822	0	18.1	0	0.77	5.362	96.2	2.1036	24	666	20.2	380.79	10.19	20.8
4.55587	0	18.1	0	0.718	3.561	87.9	1.6132	24	666	20.2	354.7	7.12	27.5
3.69695	0	18.1	0	0.718	4.963	91.4	1.7523	24	666	20.2	316.03	14	21.9
13.5222	0	18.1	0	0.631	3.863	100	1.5106	24	666	20.2	131.42	13.33	23.1
4.89822	0	18.1	0	0.631	4.97	100	1.3325	24	666	20.2	375.52	3.26	50
6.53876	0	18.1	1	0.631	7.016	97.5	1.2024	24	666	20.2	392.05	2.96	50
9.2323	0	18.1	0	0.631	6.216	100	1.1691	24	666	20.2	366.15	9.53	50
8.26725	0	18.1	1	0.668	5.875	89.6	1.1296	24	666	20.2	347.88	8.88	50
11.1081	0	18.1	0	0.668	4.906	100	1.1742	24	666	20.2	396.9	34.77	13.8
18.4982	0	18.1	0	0.668	4.138	100	1.137	24	666	20.2	396.9	37.97	13.8
15.288	0	18.1	0	0.671	6.649	93.3	1.3449	24	666	20.2	363.02	23.24	13.9
9.82349	0	18.1	0	0.671	6.794	98.8	1.358	24	666	20.2	396.9	21.24	13.3
9.18702	0	18.1	0	0.7	5.536	100	1.5804	24	666	20.2	396.9	23.6	11.3
7.99248	0	18.1	0	0.7	5.52	100	1.5331	24	666	20.2	396.9	24.56	12.3
20.0849	0	18.1	0	0.7	4.368	91.2	1.4395	24	666	20.2	285.83	30.63	8.8
24.3938	0	18.1	0	0.7	4.652	100	1.4672	24	666	20.2	396.9	28.28	10.5
22.5971	0	18.1	0	0.7	5	89.5	1.5184	24	666	20.2	396.9	31.99	7.4
8.15174	0	18.1	0	0.7	5.39	98.9	1.7281	24	666	20.2	396.9	20.85	11.5
5.29305	0	18.1	0	0.7	6.051	82.5	2.1678	24	666	20.2	378.38	18.76	23.2
11.5779	0	18.1	0	0.7	5.036	97	1.77	24	666	20.2	396.9	25.68	9.7
13.3598	0	18.1	0	0.693	5.887	94.7	1.7821	24	666	20.2	396.9	16.35	12.7
5.87205	0	18.1	0	0.693	6.405	96	1.6768	24	666	20.2	396.9	19.37	12.5
38.3518	0	18.1	0	0.693	5.453	100	1.4896	24	666	20.2	396.9	30.59	5
25.0461	0	18.1	0	0.693	5.987	100	1.5888	24	666	20.2	396.9	26.77	5.6
14.2362	0	18.1	0	0.693	6.343	100	1.5741	24	666	20.2	396.9	20.32	7.2
24.8017	0	18.1	0	0.693	5.349	96	1.7028	24	666	20.2	396.9	19.77	8.3
11.9511	0	18.1	0	0.659	5.608	100	1.2852	24	666	20.2	332.09	12.13	27.9
7.40389	0	18.1	0	0.597	5.617	97.9	1.4547	24	666	20.2	314.64	26.4	17.2
28.6558	0	18.1	0	0.597	5.155	100	1.5894	24	666	20.2	210.97	20.08	16.3
45.7461	0	18.1	0	0.693	4.519	100	1.6582	24	666	20.2	88.27	36.98	7
18.0846	0	18.1	0	0.679	6.434	100	1.8347	24	666	20.2	27.25	29.05	7.2
25.9406	0	18.1	0	0.679	5.304	89.1	1.6475	24	666	20.2	127.36	26.64	10.4
73.5341	0	18.1	0	0.679	5.957	100	1.8026	24	666	20.2	16.45	20.62	8.8
11.8123	0	18.1	0	0.718	6.824	76.5	1.794	24	666	20.2	48.45	22.74	8.4
8.79212	0	18.1	0	0.584	5.565	70.6	2.0635	24	666	20.2	3.65	17.16	11.7
15.8603	0	18.1	0	0.679	5.896	95.4	1.9096	24	666	20.2	7.68	24.39	8.3
37.6619	0	18.1	0	0.679	6.202	78.7	1.8629	24	666	20.2	18.82	14.52	10.9
7.36711	0	18.1	0	0.679	6.193	78.1	1.9356	24	666	20.2	96.73	21.52	11
9.33889	0	18.1	0	0.679	6.38	95.6	1.9682	24	666	20.2	60.72	24.08	9.5
10.0623	0	18.1	0	0.584	6.833	94.3	2.0882	24	666	20.2	81.33	19.69	14.1
6.44405	0	18.1	0	0.584	6.425	74.8	2.2004	24	666	20.2	97.95	12.03	16.1
5.58107	0	18.1	0	0.713	6.436	87.9	2.3158	24	666	20.2	100.19	16.22	14.3
13.9134	0	18.1	0	0.713	6.208	95	2.2222	24	666	20.2	100.63	15.17	11.7
15.1772	0	18.1	0	0.74	6.152	100	1.9142	24	666	20.2	9.32	26.45	8.7
9.39063	0	18.1	0	0.74	5.627	93.9	1.8172	24	666	20.2	396.9	22.88	12.8
22.0511	0	18.1	0	0.74	5.818	92.4	1.8662	24	666	20.2	391.45	22.11	10.5
9.72418	0	18.1	0	0.74	6.406	97.2	2.0651	24	666	20.2	385.96	19.52	17.1
5.66637	0	18.1	0	0.74	6.219	100	2.0048	24	666	20.2	395.69	16.59	18.4
9.96654	0	18.1	0	0.74	6.485	100	1.9784	24	666	20.2	386.73	18.85	15.4
12.8023	0	18.1	0	0.74	5.854	96.6	1.8956	24	666	20.2	240.52	23.79	10.8
10.6718	0	18.1	0	0.74	6.459	94.8	1.9879	24	666	20.2	43.06	23.98	11.8
9.92485	0	18.1	0	0.74	6.251	96.6	2.198	24	666	20.2	388.52	16.44	12.6
9.32909	0	18.1	0	0.713	6.185	98.7	2.2616	24	666	20.2	396.9	18.13	14.1
5.44114	0	18.1	0	0.713	6.655	98.2	2.3552	24	666	20.2	355.29	17.73	15.2
5.09017	0	18.1	0	0.713	6.297	91.8	2.3682	24	666	20.2	385.09	17.27	16.1
8.24809	0	18.1	0	0.713	7.393	99.3	2.4527	24	666	20.2	375.87	16.74	17.8
4.75237	0	18.1	0	0.713	6.525	86.5	2.4358	24	666	20.2	50.92	18.13	14.1
8.20058	0	18.1	0	0.713	5.936	80.3	2.7792	24	666	20.2	3.5	16.94	13.5
7.75223	0	18.1	0	0.713	6.301	83.7	2.7831	24	666	20.2	272.21	16.23	14.9
6.80117	0	18.1	0	0.713	6.081	84.4	2.7175	24	666	20.2	396.9	14.7	20
4.81213	0	18.1	0	0.713	6.701	90	2.5975	24	666	20.2	255.23	16.42	16.4
3.69311	0	18.1	0	0.713	6.376	88.4	2.5671	24	666	20.2	391.43	14.65	17.7
6.65492	0	18.1	0	0.713	6.317	83	2.7344	24	666	20.2	396.9	13.99	19.5
5.82115	0	18.1	0	0.713	6.513	89.9	2.8016	24	666	20.2	393.82	10.29	20.2
7.83932	0	18.1	0	0.655	6.209	65.4	2.9634	24	666	20.2	396.9	13.22	21.4
3.1636	0	18.1	0	0.655	5.759	48.2	3.0665	24	666	20.2	334.4	14.13	19.9
3.77498	0	18.1	0	0.655	5.952	84.7	2.8715	24	666	20.2	22.01	17.15	19
4.42228	0	18.1	0	0.584	6.003	94.5	2.5403	24	666	20.2	331.29	21.32	19.1
15.5757	0	18.1	0	0.58	5.926	71	2.9084	24	666	20.2	368.74	18.13	19.1
13.0751	0	18.1	0	0.58	5.713	56.7	2.8237	24	666	20.2	396.9	14.76	20.1
4.03841	0	18.1	0	0.532	6.229	90.7	3.0993	24	666	20.2	395.33	12.87	19.6
3.56868	0	18.1	0	0.58	6.437	75	2.8965	24	666	20.2	393.37	14.36	23.2
8.05579	0	18.1	0	0.584	5.427	95.4	2.4298	24	666	20.2	352.58	18.14	13.8
4.87141	0	18.1	0	0.614	6.484	93.6	2.3053	24	666	20.2	396.21	18.68	16.7
15.0234	0	18.1	0	0.614	5.304	97.3	2.1007	24	666	20.2	349.48	24.91	12
10.233	0	18.1	0	0.614	6.185	96.7	2.1705	24	666	20.2	379.7	18.03	14.6
14.3337	0	18.1	0	0.614	6.229	88	1.9512	24	666	20.2	383.32	13.11	21.4
5.82401	0	18.1	0	0.532	6.242	64.7	3.4242	24	666	20.2	396.9	10.74	23
5.70818	0	18.1	0	0.532	6.75	74.9	3.3317	24	666	20.2	393.07	7.74	23.7
2.81838	0	18.1	0	0.532	5.762	40.3	4.0983	24	666	20.2	392.92	10.42	21.8
2.37857	0	18.1	0	0.583	5.871	41.9	3.724	24	666	20.2	370.73	13.34	20.6
5.69175	0	18.1	0	0.583	6.114	79.8	3.5459	24	666	20.2	392.68	14.98	19.1
4.83567	0	18.1	0	0.583	5.905	53.2	3.1523	24	666	20.2	388.22	11.45	20.6
0.15086	0	27.74	0	0.609	5.454	92.7	1.8209	4	711	20.1	395.09	18.06	15.2
0.20746	0	27.74	0	0.609	5.093	98	1.8226	4	711	20.1	318.43	29.68	8.1
0.10574	0	27.74	0	0.609	5.983	98.8	1.8681	4	711	20.1	390.11	18.07	13.6
0.11132	0	27.74	0	0.609	5.983	83.5	2.1099	4	711	20.1	396.9	13.35	20.1
0.17331	0	9.69	0	0.585	5.707	54	2.3817	6	391	19.2	396.9	12.01	21.8
0.26838	0	9.69	0	0.585	5.794	70.6	2.8927	6	391	19.2	396.9	14.1	18.3
0.17783	0	9.69	0	0.585	5.569	73.5	2.3999	6	391	19.2	395.77	15.1	17.5
0.06263	0	11.93	0	0.573	6.593	69.1	2.4786	1	273	21	391.99	9.67	22.4
0.04527	0	11.93	0	0.573	6.12	76.7	2.2875	1	273	21	396.9	9.08	20.6
0.06076	0	11.93	0	0.573	6.976	91	2.1675	1	273	21	396.9	5.64	23.9
0.04741	0	11.93	0	0.573	6.03	80.8	2.505	1	273	21	396.9	7.88	11.9

Code để đọc và xử lý data. Với dữ liệu nhà Boston, chúng ta cần chuẩn hóa dữ liệu vì các cột dữ liệu có đơn vị dữ liệu khác nhau và các giá trị phân bố khác nhau. (Sau khi làm xong ví dụ này, các bạn hãy chạy thử chương trình khi data không được chuẩn hóa, và quan sát, so sánh kết quả chương trình gốc).

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

data = pd.read_csv('BostonHousing.csv')

def normal(x:list):
    maxi = max(x)
    mini = min(x)
    avg = np.mean(x)
    new = [(i-avg)/(maxi-mini) for i in x ]
    
    return new

df = data.copy()
df = df.apply(normal, axis=0)

Xd = df.drop(columns=['medv'])
Xd.insert(0, 'X0', 1) # bias 

# numpy array format
y = df.medv.values
X = Xd.values

# sample size
m = len(df.index)
n = X.shape[1]
theta = np.ones(n)

Cài theo cách dùng chỉ mục index

# No Vectorization - batch gradien descent

# cost function
def cost_loop(theta = theta, x=X, y=y, m=m, n=n):
    cost = 0
    for i in range(m):
        hypo_i = 0
        for j in range(n):
            hypo_i += theta[j]*X[i,j]
        cost_i = (hypo_i - y[i])**2
        cost += cost_i
    cost = (1/m)*cost
    
    return cost

# training
learning_rate = 0.01
theta = np.ones(n)
cost_list = []

for itr in range(500):    
    dev_list = []
    for k in range(n):
        dev_sum = 0
        for i in range(m):
            hypo_i = 0
            for j in range(n):
                hypo_i += theta[j]*X[i,j]
            dev_i = (hypo_i - y[i])*X[i,k]
            dev_sum += dev_i
        dev_sum = (2/m)*dev_sum

        dev_list.append(dev_sum)
    
    theta = theta - learning_rate*np.array(dev_list)    
    cost_val = cost_loop(theta)    
    cost_list.append(cost_val)
    

plt.plot(np.arange(0, 500),cost_list)
plt.xlabel('epoch')
plt.ylabel('Giá trị loss')

Giá trị loss qua các vòng lặp

Cài theo phương pháp vectorization

# Vectorization

# Initialize theta
theta = np.ones(n)

def cost(theta, X=X, y=y, m=m):
    cost = np.dot(np.dot(X,theta) - y, np.dot(X,theta) - y)
    cost = (1/m)*cost
    return cost

# learning rate
learning_rate = 0.01
theta = np.ones(n)
cost_list = []

for i in range(500):  
    output = np.dot(X,theta)
    loss_grd = output - y
    gradients = (2/m)*np.dot(np.transpose(X), loss_grd)
    theta = theta - learning_rate*gradients       
    cost_val = cost(theta)
    cost_list.append(cost_val)

plt.plot(np.arange(0, 500),cost_list)
plt.xlabel('epoch')
plt.ylabel('Giá trị loss')

Giá trị loss qua các vòng lặp

Ở ví dụ này, khi chạy hai đoạn code để huấn luyện mô hình, chúng ta dễ dạng nhận ra cách cài đặt theo vectorization nhanh hơn rất nhiều so với cách dùng chỉ mục index thông thường.

Thật may mắn là chúng ta không cần phải là siêu sao coding để cài được những chương trình dùng kỹ thuật vectorization. Các thư viện phổ biến hiện nay đã làm những phần việc này. Khi các bạn muốn xây dựng một hàm hay thực hiện một tác vụ nào đó, các bạn nên dùng những hàm có sẵn trong thư viện (nếu có) vì code chúng ta cài thường tệ hơn code trong các thư viện ^^.

Prev Linear regression

Next Stochastic, mini-batch, và batch gradient descent

AI Cho Mọi Người

Tuần 1 - Ngày 10 tháng 7 năm 2019

Tuần 2 - Ngày 17 tháng 7 năm 2019

Tuần 3 - Ngày 24 tháng 7 năm 2019

Tuần 4 - Ngày 31 tháng 7 năm 2019

Tuần 5 - Ngày 7 tháng 8 năm 2019

Advanced Python

Tuần 6 - Ngày 14 tháng 8 năm 2019

Tuần 7 - Ngày 28 tháng 8 năm 2019

Tuần 8

Tuần 9

AI Cho Mọi Người

Vectorization