过拟合与欠拟合欠拟合模型太简单没学到数据的规律过拟合模型太复杂把噪声也当成规律记住了训练集、验证集、测试集训练集用来训练模型验证集用来调参和选模型测试集最后只用来评估一次检验模型真实水平交叉验证交叉验证就是把数据分成几份轮流用其中一份当验证集其他当训练集最后取平均从何而来过拟合与欠拟合欠拟合模型复杂度不够比如用直线去拟合曲线。过拟合模型复杂度太高参数太多把噪声也学进去了。代码import numpy as npimport matplotlib.pyplot as pltfrom sklearn.linear_model import LinearRegressionfrom sklearn.preprocessing import PolynomialFeaturesfrom sklearn.pipeline import make_pipelinefrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import mean_squared_error生成带噪声的数据np.random.seed(42)n 100X np.linspace(-3, 3, n).reshape(-1, 1)y X.ravel() ** 3 - 3 * X.ravel() np.random.normal(0, 2, n)划分训练集和测试集X_train, X_test, y_train, y_test train_test_split(X, y, test_size0.3, random_state42)degrees [1, 3, 15] # 不同复杂度plt.figure(figsize(15, 4))for i, degree in enumerate(degrees):model make_pipeline(PolynomialFeatures(degree), LinearRegression())model.fit(X_train, y_train)y_train_pred model.predict(X_train) y_test_pred model.predict(X_test) train_mse mean_squared_error(y_train, y_train_pred) test_mse mean_squared_error(y_test, y_test_pred) plt.subplot(1, 3, i 1) plt.scatter(X_train, y_train, colorblue, alpha0.4, label训练数据) plt.scatter(X_test, y_test, colorgreen, alpha0.4, label测试数据) # 画预测曲线 X_line np.linspace(-3, 3, 300).reshape(-1, 1) y_line model.predict(X_line) plt.plot(X_line, y_line, colorred, label模型曲线) plt.title(f次数 {degree}\n训练MSE {train_mse:.2f}, 测试MSE {test_mse:.2f}) plt.legend() plt.grid(True)plt.tight_layout()plt.show()训练集、验证集、测试集早期人们只用训练集和测试集。后来发现调参的时候如果反复看测试集测试集就相当于被“污染”了。于是又切出一个验证集专门用来调参代码生成一个二分类数据集X, y make_classification(n_samples1000, n_features10, random_state42)第一步先切出测试集X_train_val, X_test, y_train_val, y_test train_test_split(X, y, test_size0.2, random_state42)第二步再从训练集里切出验证集X_train, X_val, y_train, y_val train_test_split(X_train_val, y_train_val, test_size0.25, random_state42)print(f训练集大小: {len(X_train)}“)print(f验证集大小: {len(X_val)}”)print(f测试集大小: {len(X_test)})模拟调参在训练集上训练在验证集上选择最好的 C 参数best_c Nonebest_val_acc 0for c in [0.001, 0.01, 0.1, 1, 10, 100]:model LogisticRegression(Cc, max_iter1000)model.fit(X_train, y_train)val_acc accuracy_score(y_val, model.predict(X_val)) print(fC {c:6} 验证集准确率 {val_acc:.3f}) if val_acc best_val_acc: best_val_acc val_acc best_c cprint(f\n最佳 C {best_c}, 验证集准确率 {best_val_acc:.3f})最后在测试集上评估final_model LogisticRegression(Cbest_c, max_iter1000)final_model.fit(X_train, y_train)test_acc accuracy_score(y_test, final_model.predict(X_test))print(f测试集准确率 {test_acc:.3f})交叉验证因为有时候数据太少单独切一个验证集验证集可能很不稳定。比如你只有100条数据切20%当验证集验证集只有20条结果可能随机性很大。交叉验证就是为了解决这个“验证集太小”的问题。代码import numpy as npfrom sklearn.datasets import make_classificationfrom sklearn.linear_model import LogisticRegressionfrom sklearn.model_selection import cross_val_score, KFoldfrom sklearn.metrics import accuracy_score生成一个二分类数据集X, y make_classification(n_samples500, n_features10, random_state42)model LogisticRegression(max_iter1000)5 折交叉验证kf KFold(n_splits5, shuffleTrue, random_state42)scores cross_val_score(model, X, y, cvkf, scoring‘accuracy’)print(“每一折的准确率”, np.round(scores, 3))print(f平均准确率{scores.mean():.3f}“)print(f标准差{scores.std():.3f}”)过拟合与欠拟合它们是用来判断“模型是不是合适的”。你的目标永远是找一个中间状态既不太简单也不太复杂。训练集、验证集、测试集防止你“自欺欺人”。用验证集调参用测试集最终检验这样你得到的评估结果才真实可信。交叉验证:让模型评估更稳定、更可靠尤其是数据量少的时候。