University of Birmingham > Talks@bham > Artificial Intelligence and Natural Computation seminars > Cost-sensitive Boosting, Margin Maximization and Information Theory

Cost-sensitive Boosting, Margin Maximization and Information Theory

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Hector Basevi.

Host: Prof. Ata Kaban

Speaker’s website:

Abstract: In the first part of the talk, we provide a unifying perspective for two decades of work on cost-sensitive Boosting algorithms. We critique the relevant literature – consisting of more than 15 variants of the original algorithm – using four theoretical frameworks: decision theory, functional gradient descent, margin theory, and probabilistic modelling. We find that only 3 of the published Adaboost variants are consistent with the rules of all the frameworks — and even they require their outputs to be calibrated to achieve this. Experiments on 18 datasets across 21 degrees of imbalance support the hypothesis – showing that once calibrated, they perform equivalently, and outperform all others. Our final recommendation – based on simplicity, flexibility and performance – is to use the original Adaboost algorithm with a shifted decision threshold and calibrated probability estimates.

The reason for the poor calibration of the scores generated by Boosted classifiers lies in the margin-maximization property of Boosting, which forces the ensemble to be overconfident in its predictions. In the second part of the talk, we focus on the more positive aspect of this property: constructing classifiers good at generalization. We then discuss ongoing work on interpreting margin maximization from an information-theoretic perspective and connecting it to recent insights behind the success of Deep Neural Networks.

This talk is part of the Artificial Intelligence and Natural Computation seminars series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


Talks@bham, University of Birmingham. Contact Us | Help and Documentation | Privacy and Publicity.
talks@bham is based on from the University of Cambridge.