Python You can import any data you want. Using default hyperparameters: 1. Construct Naive Bayes (NB) models on the training set.
Python
You can import any data you want.
Using default hyperparameters:
1. Construct Naive Bayes (NB) models on the training set.
2. Calculate the confusion matrix and report the following performance metrics on the training set: Accuracy, F1 Score, AUC, Sensitivity, Specificity, and Precision.
You can use the function p1_metrics for this purpose.
3. Calculate the same metrics by applying the trained model to the validation set. Compare and contrast the errors each model makes in terms of each class.
#peformance metric functions
from sklearn.metrics import confusion_matrix, roc_auc_score, f1_score
import numpy as np
#A list of keys for the dictionary returned by p1_metrics
metric_keys = ['auc','f1','accuracy','sensitivity','specificity', 'precision']
def p1_metrics(y_true,y_pred,include_cm=True):
cm = confusion_matrix(y_true,y_pred)
tn, fp, fn, tp = cm.ravel()
if include_cm:
return {
'auc': roc_auc_score(y_true,y_pred),
'f1': f1_score(y_true,y_pred),
'accuracy': (tp+tn)/np.sum(cm),
'sensitivity': tp/(tp+fn),
'specificity': tn/(tn+fp),
'precision': tp/(tp+fp),
'confusion_matrix': cm}
else:
return {
'auc': roc_auc_score(y_true,y_pred),
'f1': f1_score(y_true,y_pred),
'accuracy': (tp+tn)/np.sum(cm),
'sensitivity': tp/(tp+fn),
'specificity': tn/(tn+fp),
'precision': tp/(tp+fp)}
#This wrapper can be used to return multiple performance metrics during cross-validation
def p1_metrics_scorer(clf,X,y_true):
y_pred=clf.predict(X)
return p1_metrics(y_true,y_pred,include_cm=False)
Step by step
Solved in 3 steps
