机器学习中的训练模型评估方法和性能度量

机器学习实验过程,是我们希望通过训练集能够学习出在新样本(测试集)上也表现良好的学习期(模型)。我们常用“错误率”(error rate)或“精度”(accuracy)来量化模型的…

机器学习中的训练模型评估方法和性能度量

机器学习实验过程,是我们希望通过训练集能够学习出在新样本(测试集)上也表现良好的学习期(模型)。我们常用“错误率”(error rate)或“精度”(accuracy)来量化模型的好坏程度,其中错误率 = 1 – 精度。

一般地,我们用“误差”表示学习器的实际预测输出与样本的真是输出之间的差异,在训练集上的误差称为“训练误差”,在新样本(测试集)上的误差称为“泛化误差”。

显然地,我们希望“训练误差”和“泛化误差”都尽可能的小。这就需要权衡,从而引发了机器学习中最难解决的重大问题——过拟合(overfitting)和欠拟合(underfitting)。

怎么解释呢:

因为我们在训练模型的时候并不能准确的知道测试集的样子,而只能通过训练集来学习。那我们应该使得模型满足训练集在一个什么程度才能更好的适应新样本呢?

如果我们做到了100%的匹配了训练集,那么因为新样本的不可预知性,往往对于新来的样本就达不到很好的效果,更因为过拟合导致的多类分界线过于“神奇”,错误往往更多发生;反之如果对训练集不是那么苛刻的要求匹配比率,那么模型的正确性又见得能的到太好的保证,也就是欠拟合。

机器学习中的训练模型评估方法和性能度量

那么,在我们机器学习的实验测试中,就需要去科学的评估学习器。这既是为了衡量当前模型的学习水平,同时也是在多个模型中做出选择的最佳方式。

在我们进行的项目中,情况一般都是甲方给出数据,而我们根据给定的数据训练一个良好的预测模型。在模型评估阶段,学习器的评估方法就有以下几种:

1.留出法

顾名思义,就是讲训练集留出一部分当做测试集进行测试。这在工程中,在过“脏”的数据情景下是很常用的,因为此时数据清洗工作真的十分困难,评估工作往往从简。但不是说这个方法不好,从概率学角度来说,也是存在科学道理的,一般训练集:测试集 = 7 : 3。但个人觉得不如第二种方法。

2.交叉验证法

交叉验证法指的是将数据等分为N份,进行N次验证实验。每一次验证实验就取这N份中的1份当做测试集,其它N-1份当做训练集。根据经验来说,N一般取10,以平衡评估工作的工作量。

3.自助法

据说自助法在数据集较小、难以划分训练/测试集时很有用。

但是实际工程中真的很少用到,它的过程在我理解其实就是一个根据概率知识自行制造训练集的过程,其过程大致如下:对初始数据集进行有放回采样m次得到一个大小空间是m的训练集,这个训练集是有重复的训练集,然后原始数据集没被采样过的样本当作测试集,这个过程根据概率来讲大概有36%的样本没有被采样到。比例也很不错。可以用来做评估实验。

4.调参法

调参法就是在学习器中和某些参数相关时,通过调参来一步步找到模型效果最好的极值。通过跳跃区间取参,也有可能找到接近最好的值。

机器学习中的训练模型评估方法和性能度量

评估工作除了上述实验估计的方法外,也需要定量的评价模型的泛化能力,这就是性能度量。

性能度量包括以下指标,我会着重讲述查准率和查全率,这也是我们最常用的量化标准:

1.错误率和精度

错误率和精度就是直接预测的错误率和正确率。

2.准确率和召回率

机器学习中的训练模型评估方法和性能度量

如图所示,准确率指预测结果中准确的比例,召回率指预测结果正例占总正例的比例。F-measure是非常常用的性能度量指标。F度量指标的意义大致是认为当查准率等于查全率时,模型效果最好。

3.ROC和AUC

ROC曲线和AUC也是能够衡量模型性能的重要指标,其中ROC曲线是真正例率和假正例率的二维曲线,AUC是其面积,可以通过AUC面积比较来比较模型的性能相对好坏。在我们的工程和学习研究中很少遇到,如需要交流可以留言。

4.代价敏感错误率与代价曲线

利用正例概率代价和归一化代价二维曲线的面积比较,在日常项目中不会使用,深入研究下可能需要了解。

This is cheap nfl jerseys most common in the stock and property market. In fact, this May we will custom jerseys end the season as the number one network in viewers and 25 to 54 and despite not having the Super Bowl this year, we are very much in the running for the 18 to 49 crown as Baratas Ray Ban well.. You build your body up and get it strong to take the pounding. When David wrote God’s Word, no extraneous thoughts or ideas crept in.. Among the items to look for when handicapping NFL Preseason games is to assess the intentions of the head coach. It’s not like they are unaware that people are offended, either. You want to make sure those are good and tight as well. Big time partying is an integral part of the IPL show. If Blount isn’t one of your favorite Patriots right now, I have to assume he trampled a member of your family who tried to tackle him at some point. Going over the top at the Battle of the Somme (1916), a few British soldiers kicked soccer balls in the general Baratas Replicas Ray Ban direction of the German lines. She’s given us beautiful turquoise oceans, tumbling waterfalls, rainbows, rocky red cliffs and rolling, green fields. Snell was Cheap Jerseys recalled from AAA and he is set to take the hill tomorrow against the Seattle Mariners. The largest series to date, however, was that of Corsellis et al,5 who described findings in 15 boxers. Thank you for joining us today. Some of those hits are admittedly tough to call, but the four below are not. The automotive, consumer electronics, construction, clothing and brewing industries are examples of light industrial businesses.. Notable achievements of Konstandinos include winning gold medals in the Summer Olympics wholesale nfl jerseys (2000), World Championships in Athletics (2000) and the European Athletics Championships (2002). Each song serves as a chapter in an overarching story about a blind, deaf, and mute boy who becomes a religious leader (we didn’t say it was a good story). 2 deep coverage. With 1.7 million players joining the beta on PS4 and PC and logging more than 4.2 million session hours over two weeks, the beta delivered a very stable experience that also helped the Visceral Games team gather valuable player feedback.. Inquiry: What motivates you in your work? Are you in the right career? A solid work ethic is a building block for any career. Vice versa, a hook would be a ball that hooked way left, curved a lot. Really, they’re only good at killing unarmed people running scared through dimly lit corridors. Setting aside, for the present, the question, whether ray ban aviator sunglasses for men there was really any arsenic on the crumbs at all, ray ban 3217 it would clearly have been showing the unfortunate maid of womens ray ban aviators small all work no more than common justice to have allowed coaches her the opportunity of jordan sale speaking before the bread was carried discount oakley sunglasses online away.It was now seven o clock in the evening.

为您推荐

发表评论

电子邮件地址不会被公开。 必填项已用*标注

返回顶部
本页面加载共:43 次查询 | 用时 0.699 秒 | 消耗 53.56MB 内存