{"id":20901,"date":"2024-02-20T22:17:06","date_gmt":"2024-02-20T21:17:06","guid":{"rendered":"https:\/\/complex-systems-ai.com\/?page_id=20901"},"modified":"2024-02-20T23:00:48","modified_gmt":"2024-02-20T22:00:48","slug":"auc-et-roc","status":"publish","type":"page","link":"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/","title":{"rendered":"AUC and ROC"},"content":{"rendered":"<div data-elementor-type=\"wp-page\" data-elementor-id=\"20901\" class=\"elementor elementor-20901\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-63eb5d7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"63eb5d7\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-e7039b1\" data-id=\"e7039b1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7dd8f10 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"7dd8f10\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Supervised learning<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-1c41586\" data-id=\"1c41586\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1a6223a elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"1a6223a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/en\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Home page<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-a1d3f6a\" data-id=\"a1d3f6a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-dea1211 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"dea1211\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/www.jair.org\/index.php\/jair\/article\/view\/12228\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Wiki<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-16684ff elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"16684ff\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d7b4bfe\" data-id=\"d7b4bfe\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-492ad57 elementor-widget elementor-widget-heading\" data-id=\"492ad57\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/#Courbe-AUC-et-ROC-interpretation-et-multiclasse\" >AUC and ROC curve, interpretation and multiclass<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/#Mesures-de-performance\" >Performance measures<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/#Calcul-du-ROC\" >ROC calculation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/#Interpretation-mathematique\" >Mathematical interpretation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/complex-systems-ai.com\/en\/learning-supervises\/auc-and-rock\/#Courbe-AUC-ROC-pour-le-probleme-mutliclasse\" >AUC - ROC curve for the multiclass problem<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Courbe-AUC-et-ROC-interpretation-et-multiclasse\"><\/span>AUC and ROC curve, interpretation and multiclass<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5d87fc6 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5d87fc6\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b6862d9\" data-id=\"b6862d9\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1a210b4 elementor-widget elementor-widget-text-editor\" data-id=\"1a210b4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This tutorial presents the AUC and ROC curve as well as how to interpret the results. The multiclass case is also presented.<\/p><p><img decoding=\"async\" class=\"aligncenter wp-image-11096 size-full\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2020\/09\/cropped-Capture.png\" alt=\"AUC and ROC curve\" width=\"97\" height=\"97\" title=\"\"><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b6de630 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b6de630\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f623007\" data-id=\"f623007\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-99b7ca6 elementor-widget elementor-widget-heading\" data-id=\"99b7ca6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Mesures-de-performance\"><\/span>Performance measures<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-2688208 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"2688208\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-73a9255\" data-id=\"73a9255\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-741c5dd elementor-widget elementor-widget-text-editor\" data-id=\"741c5dd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In Machine Learning, measuring performance is an essential task. So when it comes to a classification problem, we can rely on an AUC \u2013 ROC Curve. When we need to check or visualize the performance of the multi-class classification problem, we use the AUC (Area Under The Curve) ROC (Receiver Operating Characteristics) curve. This is one of the most important evaluation metrics to check the performance of any classification model. It is also written AUROC (Area Under the Receiver Operating Characteristics).<\/p><p>The AUC\u2013ROC curve is a performance measure for classification problems at different threshold settings. ROC is a probability curve and AUC represents the degree or measure of separability. It indicates to what extent the model is able to distinguish classes. The higher the AUC, the more the model is able to predict 0 classes as 0 and 1 class as 1. By analogy, the higher the AUC, the more the model is able to distinguish patients with the disease from those who do not have.<\/p><p>The ROC curve is plotted with TPR=TP\/(TP+FN) versus FPR=1-Specificity=FP\/(TN+FP) where TPR is on the y-axis and FPR is on the x-axis.<\/p><p><img decoding=\"async\" class=\"alignnone size-full wp-image-20907\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc1.png\" alt=\"ROC-AUC\" width=\"220\" height=\"220\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc1.png 220w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc1-150x150.png 150w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc1-12x12.png 12w\" sizes=\"(max-width: 220px) 100vw, 220px\" \/><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-28edac2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"28edac2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2dbbb37\" data-id=\"2dbbb37\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-93dc874 elementor-widget elementor-widget-heading\" data-id=\"93dc874\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Calcul-du-ROC\"><\/span>ROC calculation<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f6dbb4c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f6dbb4c\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fa1ca21\" data-id=\"fa1ca21\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6ed509b elementor-widget elementor-widget-text-editor\" data-id=\"6ed509b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Sensitivity and specificity (TPR and FPR) are inversely proportional to each other. So, as we increase sensitivity, specificity decreases, and vice versa.<\/p><p>When we train a classification model, we get the probability of getting a result. In this case, our example will be the probability of repaying a loan.<\/p><p><img fetchpriority=\"high\" decoding=\"async\" class=\"alignnone wp-image-20908 size-full\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc2.webp\" alt=\"AUC-ROC\" width=\"720\" height=\"668\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc2.webp 720w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc2-300x278.webp 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc2-13x12.webp 13w\" sizes=\"(max-width: 720px) 100vw, 720px\" \/><\/p><p>The probabilities vary between 0 and 1. The higher the value, the more likely the person is to repay a loan.<\/p><p>The next step is to find a threshold to classify the probabilities as \u201cwill refund\u201d or \u201cwill not refund\u201d.<\/p><p>In the example in the figure, we have selected a threshold of 0.35 (the classification models will automatically select the value giving the best precision):<\/p><ul><li>All predictions at or above this threshold are classified as &quot;will refund&quot;<\/li><li>All predictions below this threshold are classified as &quot;will not refund&quot;<\/li><\/ul><p>We then examine which of these predictions were correctly classified or misclassified. With such information we can construct a confusion matrix.<\/p><p>At this point we have<\/p><ul><li>correctly classified 90% of all positives, those who have \u201crepaid\u201d (TPR)<\/li><li>40 % of all negatives were misclassified, those that &quot;did not refund&quot; (FPR)<\/li><\/ul><p>We can notice that the results for TPR and FPR decrease as the threshold increases. If we look at the first, where the threshold is 0:<\/p><ul><li>All positives have been correctly classified, so TPR = 100 %<\/li><li>All negatives were misclassified, so FPR = 100 %<\/li><\/ul><p>In the last example graph, where the threshold is 1:<\/p><ul><li>All positives were misclassified, so TPR = 0 %<\/li><li>All negatives were correctly classified, so FPR = 0 %<\/li><\/ul><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-20909 size-large\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-1024x417.webp\" alt=\"AUC-ROC\" width=\"1024\" height=\"417\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-1024x417.webp 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-300x122.webp 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-768x313.webp 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-1536x626.webp 1536w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-2048x835.webp 2048w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc3-18x7.webp 18w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>To plot the ROC curve, we need to calculate the TPR and FPR for many different thresholds (this step is included in all relevant libraries under the name scikit-learn).<\/p><p>For each threshold, we plot the FPR value on the x-axis and the TPR value on the y-axis. We then join the points with a line. That&#039;s it!<\/p><p>Below in the figure below we can see how each point on the ROC curve represents the FPR and TRP of a classification at a given threshold.<\/p><p>Notice how the threshold at 1 leads to the first point at (0, 0) and the threshold at 0 leads to the last point at (1, 1).<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-20910 size-large\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-1024x529.webp\" alt=\"AUC-ROC\" width=\"1024\" height=\"529\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-1024x529.webp 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-300x155.webp 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-768x397.webp 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-1536x794.webp 1536w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-2048x1058.webp 2048w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc4-18x9.webp 18w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>The area covered below the line is called the Area Under the Curve (AUC). This is used to evaluate the performance of a classification model. The higher the AUC, the better the model is at distinguishing classes.<\/p><p>This means that in an ideal world we would like to see our line cover most of the upper left corner of the chart to achieve a higher AUC.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f2c187d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f2c187d\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5172dbc\" data-id=\"5172dbc\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-18f768b elementor-widget elementor-widget-heading\" data-id=\"18f768b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Interpretation-mathematique\"><\/span>Mathematical interpretation<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b5db85a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b5db85a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-44a4a71\" data-id=\"44a4a71\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-917c603 elementor-widget elementor-widget-text-editor\" data-id=\"917c603\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>As we know, ROC is a probability curve. Let us therefore plot the distributions of these probabilities:<\/p><p>Note: The red distribution curve is of the positive class (patients with disease) and the green distribution curve is of the negative class (patients without disease).<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-20911 size-full\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc5.webp\" alt=\"AUC-ROC\" width=\"528\" height=\"229\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc5.webp 528w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc5-300x130.webp 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc5-18x8.webp 18w\" sizes=\"(max-width: 528px) 100vw, 528px\" \/><\/p><p>In this scenario, a regression finds a clear distinction between the two classes. In the case of a decision tree, a single split is enough to have 100% success! Here the AUC is 1, which would give the following curve:<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-20912\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc6-300x273.webp\" alt=\"AUC-ROC\" width=\"300\" height=\"273\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc6-300x273.webp 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc6-13x12.webp 13w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc6.webp 365w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p><p>It&#039;s an ideal situation. When two curves do not overlap at all, it means that the model has an ideal measure of separability. It is perfectly capable of distinguishing the positive class from the negative class.<\/p><p>When two distributions overlap, we introduce type 1 and type 2 errors. Depending on the threshold, we can minimize or maximize them. When the AUC is 0.7, it means that there is a 70 % chance that the model will be able to distinguish between a positive class and a negative class.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-20913 size-large\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7-1024x332.png\" alt=\"AUC-ROC\" width=\"1024\" height=\"332\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7-1024x332.png 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7-300x97.png 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7-768x249.png 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7-18x6.png 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc7.png 1134w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>This is the worst situation. When the AUC is around 0.5, the model has no discrimination ability to distinguish the positive class from the negative class. This amounts to random prediction.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-20914 size-large\" src=\"http:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8-1024x389.png\" alt=\"AUC-ROC\" width=\"1024\" height=\"389\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8-1024x389.png 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8-300x114.png 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8-768x292.png 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8-18x7.png 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2024\/02\/Roc8.png 1049w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>When the AUC is around 0, the model actually reciprocates the classes. This means that the model predicts a negative class as a positive class and vice versa.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-dc1c125 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"dc1c125\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-828a1eb\" data-id=\"828a1eb\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7c938f3 elementor-widget elementor-widget-heading\" data-id=\"7c938f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Courbe-AUC-ROC-pour-le-probleme-mutliclasse\"><\/span>AUC - ROC curve for the multiclass problem<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0a6956b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0a6956b\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-566566e\" data-id=\"566566e\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-cce3742 elementor-widget elementor-widget-text-editor\" data-id=\"cce3742\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In a multi-class model, we can plot N number of AUC ROC curves for N number classes using the One vs ALL methodology. So, for example, if you have three classes named X, Y, and Z, you will have one ROC for X ranked against Y and Z, another ROC for Y ranked against X and Z, and the third of Z ranked with respect to Y and X.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Supervised learning Home page Wiki AUC and ROC curve, interpretation and multiclass This tutorial presents the AUC and ROC curve as well as how to interpret\u2026 <\/p>","protected":false},"author":1,"featured_media":0,"parent":20741,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-20901","page","type-page","status-publish","hentry"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/20901","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/comments?post=20901"}],"version-history":[{"count":4,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/20901\/revisions"}],"predecessor-version":[{"id":20917,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/20901\/revisions\/20917"}],"up":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/20741"}],"wp:attachment":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/media?parent=20901"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}