Classification Metrics Can't Handle A Mix Of Binary And Continuous Targets

I try to train and test several scikit-learn models and attempt to print off the accuracy. Only some of these models work, others fail with the ValueError: Classification metrics c

Solution 1:

All your commented-out models are not classifiers but regression models, for which accuracy is meaningless.

You get the error because these regression models do not produce binary outcomes, but continuous (float) numbers (as all regression models do); so, when scikit-learn attempts to calculate the accuracy by comparing a binary number (true label) with a float (predicted value), it not unexpectedly gives an error. And this cause is clearly hinted at the error message itself:

Classification metrics can't handle a mix of binary and continuous target

Solution 2:

I have used a few models for stacking using the vecstack and set needs_proba=True and then got this error. I solved it by changing the metric inside the stacking. because stacking use class prediction by default, so in case you want to have probabilities you should change the metric as well. I have defined a new function as metric:

defget_classification_metric(testy, probs):
    from sklearn.metrics import precision_recall_curve
    precision, recall, thresholds = precision_recall_curve(testy, probs[:,1])
    # convert to f score
    fscore = (2 * precision * recall) / (precision + recall)
    # locate the index of the largest f score
    ix = np.argmax(fscore)
    return fscore[ix]

This function finds the highest F1 score at optimal threshold. So only need to set metric=get_classification_metric inside the stacking function.