{"id":15734,"date":"2022-04-23T16:34:31","date_gmt":"2022-04-23T15:34:31","guid":{"rendered":"https:\/\/complex-systems-ai.com\/?page_id=15734"},"modified":"2022-04-23T17:37:28","modified_gmt":"2022-04-23T16:37:28","slug":"bonnes-pratiques-de-lanalyse-exploratoire-des-donnees","status":"publish","type":"page","link":"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/good-practices-for-exploratory-data-analysis\/","title":{"rendered":"Best practices for exploratory data analysis"},"content":{"rendered":"<div data-elementor-type=\"wp-page\" data-elementor-id=\"15734\" class=\"elementor elementor-15734\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d30f757 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d30f757\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-425e02b\" data-id=\"425e02b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e9c359d elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"e9c359d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Descriptive analysis<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-b5e9124\" data-id=\"b5e9124\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e8b604f elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"e8b604f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/en\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Home page<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-a11f34a\" data-id=\"a11f34a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-818b9ec elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"818b9ec\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/en.wikipedia.org\/wiki\/Descriptive_statistics\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Wiki<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9ad47e1 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9ad47e1\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d4e86eb\" data-id=\"d4e86eb\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a69827d elementor-widget elementor-widget-text-editor\" data-id=\"a69827d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This page describes best practices for exploratory data analysis: what to do with a dataset in order to understand its content.<\/p><p><img decoding=\"async\" class=\"aligncenter wp-image-11096 size-full\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2020\/09\/cropped-Capture.png\" alt=\"best practices for exploratory data analysis\" width=\"97\" height=\"97\" title=\"\"><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-73d2f5c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"73d2f5c\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-0a3558c\" data-id=\"0a3558c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2de95aa elementor-widget elementor-widget-heading\" data-id=\"2de95aa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/good-practices-for-exploratory-data-analysis\/#Conseils-et-bonnes-pratiques-de-lanalyse-exploratoire-des-donnees-EDA\" >Exploratory Data Analysis (EDA) Tips and Best Practices<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/good-practices-for-exploratory-data-analysis\/#Description-des-valeurs-quantitatives\" >Description of quantitative values<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/good-practices-for-exploratory-data-analysis\/#Selection-des-colonnes\" >Selection of columns<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/complex-systems-ai.com\/en\/descriptive-analysis\/good-practices-for-exploratory-data-analysis\/#Donnees-qualitative\" >Qualitative data<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Conseils-et-bonnes-pratiques-de-lanalyse-exploratoire-des-donnees-EDA\"><\/span>Exploratory Data Analysis (EDA) Tips and Best Practices<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8c4f593 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8c4f593\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-291f373\" data-id=\"291f373\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-42b04dc elementor-widget elementor-widget-text-editor\" data-id=\"42b04dc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Exploratory data analysis refers to the critical process of performing initial investigations of data to discover patterns, spot anomalies, test hypotheses, and verify hypotheses using summary statistics and representations graphics.<\/p><p>It&#039;s a good practice to understand the data first and try to get as much information out of it as possible. EDA is about making sense of the data in hand, before dirtying it with it.<\/p><p>I will take an example white variant of the Wine Quality dataset which is available on UCI Machine Learning Repository and try to grab as much information from the dataset using EDA.<\/p><p>To start, I imported the necessary libraries (for this example pandas, numpy, matplotlib and seaborn) and loaded the dataset.<\/p><p><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter wp-image-15740 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-1024x226.png\" alt=\"\" width=\"1024\" height=\"226\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-1024x226.png 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-300x66.png 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-768x169.png 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-18x4.png 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA-600x132.png 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_8KJeO0SOlhyxlvXkjbvjlA.png 1121w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>I found out the total number of rows and columns in the dataset using &#039;.shape&#039;.<\/p><p>The dataset includes 4898 observations and 12 features. One of which is a dependent variable and the other 11 are independent variables \u2013 physico-chemical characteristics.<\/p><p>It is also good practice to know the columns and their corresponding data types, as well as to determine whether or not they contain null values.<\/p><p><img decoding=\"async\" class=\"aligncenter wp-image-15741 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-1024x322.png\" alt=\"\" width=\"1024\" height=\"322\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-1024x322.png 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-300x94.png 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-768x242.png 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-18x6.png 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w-600x189.png 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_7cSVoZ4OjozPG9vo-yjB-w.png 1132w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>The data has only float and integer values. No variable columns have null\/missing values.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e223078 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e223078\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-cc77079\" data-id=\"cc77079\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a558951 elementor-widget elementor-widget-heading\" data-id=\"a558951\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Description-des-valeurs-quantitatives\"><\/span>Description of quantitative values<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-25de581 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"25de581\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5276eff\" data-id=\"5276eff\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-099020f elementor-widget elementor-widget-text-editor\" data-id=\"099020f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The describe() function in pandas is very handy for getting various summary statistics. This function returns the count, mean, standard deviation, minimum and maximum values, and quantiles of the data.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15742 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-1024x317.png\" alt=\"\" width=\"1024\" height=\"317\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-1024x317.png 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-300x93.png 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-768x238.png 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-18x6.png 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w-600x186.png 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_79E_z0OZsqFExkRHm_OM5w.png 1146w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>Here, as you can see, the mean value is lower than the median value of each column which is represented by 50 % (50th percentile) in the index column. In particular, there is a large difference between the 75th %tile and max values of the \u201cresidual sugar\u201d, \u201cfree sulfur dioxide\u201d, \u201ctotal sulfur dioxide\u201d predictors. Thus, observations 1 and 2 suggest that there are extreme-outliers in our data set.<\/p><p>Python has a visualization library, Seaborn, which builds on matplotlib. It provides very attractive statistical graphs in order to perform univariate and multivariate analyses.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-39e0e52 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"39e0e52\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-3c31eab\" data-id=\"3c31eab\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f6d2b7e elementor-widget elementor-widget-heading\" data-id=\"f6d2b7e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Selection-des-colonnes\"><\/span>Selection of columns<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5948054 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5948054\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-724fd64\" data-id=\"724fd64\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f6976fb elementor-widget elementor-widget-text-editor\" data-id=\"f6976fb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>To use the data for modeling, it is necessary to remove correlated variables to improve your model. One can find correlations using pandas &#039;.corr()&#039; function and visualize the matrix of <a href=\"https:\/\/complex-systems-ai.com\/en\/correlation-and-regressions\/\">correlation<\/a> using a heatmap in seaborn.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15745 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-1024x709.jpeg\" alt=\"\" width=\"1024\" height=\"709\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-1024x709.jpeg 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-300x208.jpeg 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-768x532.jpeg 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-18x12.jpeg 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg-600x416.jpeg 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_ObHerhSsFvFV6hLSgLMYHg.jpeg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>Here we can deduce that &#039;density&#039; has a strong positive correlation with &#039;residual sugar&#039; while it has a strong negative correlation with &#039;alcohol&#039;. &quot;free sulfur dioxide&quot; and &quot;citric acid&quot; have almost no correlation with &quot;quality&quot;.<\/p><p>Since the correlation is zero, we can deduce that there is no linear relationship between these two predictors. However, it is safe to remove these features in case you apply the model of <a href=\"https:\/\/complex-systems-ai.com\/en\/correlation-and-regressions\/data-transformation-and-regression\/\">regression<\/a> linear to the data set.<\/p><p>A boxplot (or boxplot) shows the distribution of quantitative data in a way that facilitates comparisons between variables. The box shows the quartiles of the data set while the whiskers expand to show the rest of the distribution.<\/p><p>In the simplest box plot, the central rectangle extends from the first quartile to the third quartile (the interquartile range or IQR). A segment inside the rectangle shows the median, and the &quot;whiskers&quot; above and below the box show the locations of the minimum and maximum.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15746 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-1024x401.jpeg\" alt=\"\" width=\"1024\" height=\"401\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-1024x401.jpeg 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-300x117.jpeg 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-768x301.jpeg 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-18x7.jpeg 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw-600x235.jpeg 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_j1pIrOQDO9N8w27lXeTnGw.jpeg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>Outliers are either 3 \u00d7 IQR or more above the third quartile or 3 \u00d7 IQR or more below the first quartile. In our dataset, except for \u201calcohol\u201d, all other feature columns show outliers.<\/p><p>Now, to check the linearity of the variables, it is recommended to plot a distribution graph and find the asymmetry of the features. Kernel density estimation (kde) is a very useful tool for plotting the shape of a distribution.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15747 size-large\" src=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-1024x190.jpeg\" alt=\"\" width=\"1024\" height=\"190\" title=\"\" srcset=\"https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-1024x190.jpeg 1024w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-300x56.jpeg 300w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-768x143.jpeg 768w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-18x3.jpeg 18w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ-600x111.jpeg 600w, https:\/\/complex-systems-ai.com\/wp-content\/uploads\/2022\/04\/1_CrcsNnhKfN4pWgvAuO5pvQ.jpeg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p><p>The &quot;pH&quot; column appears to be distributed normally. All remaining independent variables are right-skewed\/positively skewed.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-bad09a2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"bad09a2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f538cf1\" data-id=\"f538cf1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-946c17b elementor-widget elementor-widget-heading\" data-id=\"946c17b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Donnees-qualitative\"><\/span>Qualitative data<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-91d3d39 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"91d3d39\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f082975\" data-id=\"f082975\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-877633c elementor-widget elementor-widget-text-editor\" data-id=\"877633c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>For the exploration of qualitative data, I invite you to return to the glossary of the descriptive analysis course and to choose the corresponding Exercises.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Descriptive analysis Wiki home page This page describes good practices for exploratory data analysis: what to do with a dataset in order to \u2026 <\/p>","protected":false},"author":1,"featured_media":0,"parent":15506,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-15734","page","type-page","status-publish","hentry"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/15734","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/comments?post=15734"}],"version-history":[{"count":3,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/15734\/revisions"}],"predecessor-version":[{"id":15753,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/15734\/revisions\/15753"}],"up":[{"embeddable":true,"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/pages\/15506"}],"wp:attachment":[{"href":"https:\/\/complex-systems-ai.com\/en\/wp-json\/wp\/v2\/media?parent=15734"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}