{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "16b98dba",
   "metadata": {},
   "source": [
    "# Machine learning non-supervisé et réduction de dimension \n",
    "\n",
    "Ce notebook reprend des éléments de https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html et d'autres exemples de la documentation de scikit-learn, d'après une adaptation de Rémi Gribonval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17cdd3a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c7c60bf",
   "metadata": {},
   "source": [
    "# Préliminaires: \n",
    "## Chargement de données : IRIS\n",
    "On commence comme lors de la séance 1 par charger le jeu de données IRIS. Rappelons qu'IRIS rassemble des données mesurant les dimensions (longueur, largeur) des pétales et sépales de différents échantillons de trois variétés d'iris (Setosa, Versicolour, Virginica)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b52c765",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "iris = load_iris()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47c5fa82",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = iris.data  \n",
    "y = iris.target "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71982933",
   "metadata": {},
   "source": [
    "### Question 1: Quel est le nombre $n$ d'exemples ce jeu de données ? La dimension $d$ des exemples ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8312ceab",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1991509a",
   "metadata": {},
   "source": [
    "## Conversion de format\n",
    "Par la suite il sera pratique de bénéficier des fonctionnalités de pandas utilisée lors de la Séance 2 pour visualiser le contenu de jeux de données. La fonction de conversion suivante (issue de https://stackoverflow.com/questions/38105539/how-to-convert-a-scikit-learn-dataset-to-a-pandas-dataset) pourra nous resservir."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c12c1493",
   "metadata": {},
   "outputs": [],
   "source": [
    "def sklearn_to_df(sklearn_dataset):\n",
    "    df = pd.DataFrame(sklearn_dataset.data, columns=sklearn_dataset.feature_names)\n",
    "    df['target'] = pd.Series(sklearn_dataset.target)\n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4eeb967",
   "metadata": {},
   "source": [
    "Appliquons-là à notre jeu de données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0625c002",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_iris = sklearn_to_df(iris)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06db86d1",
   "metadata": {},
   "source": [
    "### Question 2: Inspection des données\n",
    "Rappelez-vous la séance d'hier, que proposez-vous pour un premier aperçu de la structure des données ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac908135",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_iris.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37048401",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_iris.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0509e6c4",
   "metadata": {},
   "source": [
    "### Question 3: quelle variable contient la classe (le type d'iris) ? Combien de classes et combien d'exemples par classes y a-t-il ? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e849217",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "df_iris['target'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "680c0728",
   "metadata": {},
   "source": [
    "# Partie 1 - réduction de dimension\n",
    "## Première visualisation des données\n",
    "On aimerait maintenant visualiser nos données. La fonction suivante permet d'afficher un jeu de données en 2d. La couleur indique la classe (Setosa, Versicolour, Virginica)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "333f9073",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot2d(X,y,label=['first','second']): # affichage d'un tableau nx2 de données X avec des couleurs déterminées par y de taille nx1\n",
    "    x_min, x_max = X[:, 0].min() - 0.5, X[:, 0].max() + 0.5\n",
    "    y_min, y_max = X[:, 1].min() - 0.5, X[:, 1].max() + 0.5\n",
    "    plt.figure(figsize=(4, 3))\n",
    "    plt.clf()\n",
    "    # Plot the training points\n",
    "    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Set1, edgecolor=\"k\")\n",
    "    plt.xlabel(label[0])\n",
    "    plt.ylabel(label[1])\n",
    "    plt.xlim(x_min, x_max)\n",
    "    plt.ylim(y_min, y_max)\n",
    "    plt.xticks(())\n",
    "    plt.yticks(())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f5b40b8",
   "metadata": {},
   "source": [
    "Pour utiliser cette fonction on a besoin des labels des colonnes de X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0002e1e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = df_iris.columns; labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6222df4d",
   "metadata": {},
   "source": [
    "## Question 4: quel problème a-t-on pour afficher le jeu de données ? Que proposez-vous ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca714391",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plot2d(X[:,:2],y,labels[:2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7c5cda6",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot2d(X[:,2:4],y,labels[2:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91b93276",
   "metadata": {},
   "source": [
    "## Sélection de caractéristiques\n",
    "Ce qu'on vient de faire est de la réduction de dimension par *sélection de caractéristiques !*\n",
    "Pour aller plus loin (non couvert dans cette formation) il existe des méthodes automatisées de sélection de caractéristiques\n",
    "Voir https://scikit-learn.org/stable/modules/feature_selection.html pour plus de détails"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef830827",
   "metadata": {},
   "outputs": [],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f263a549",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import SelectKBest\n",
    "from sklearn.feature_selection import chi2\n",
    "X_new = SelectKBest(chi2, k=2).fit_transform(X, y)\n",
    "X_new.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a67e298c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plot2d(X_new,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5be8c62",
   "metadata": {},
   "source": [
    "-----\n",
    "## Analyse en composantes principales\n",
    "### Premier contact"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf2b258a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "X_reduced = PCA(n_components=3).fit_transform(iris.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e859946d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# To getter a better understanding of interaction of the dimensions\n",
    "# plot the first three PCA dimensions\n",
    "fig = plt.figure(1, figsize=(8, 6))\n",
    "ax = fig.add_subplot(111, projection=\"3d\", elev=-150, azim=110)\n",
    "\n",
    "ax.scatter(\n",
    "    X_reduced[:, 0],\n",
    "    X_reduced[:, 1],\n",
    "    X_reduced[:, 2],\n",
    "    c=y,\n",
    "    cmap=plt.cm.Set1,\n",
    "    edgecolor=\"k\",\n",
    "    s=40,\n",
    ")\n",
    "\n",
    "ax.set_title(\"First three PCA directions\")\n",
    "ax.set_xlabel(\"1st eigenvector\")\n",
    "ax.xaxis.set_ticklabels([])\n",
    "ax.set_ylabel(\"2nd eigenvector\")\n",
    "ax.yaxis.set_ticklabels([])\n",
    "ax.set_zlabel(\"3rd eigenvector\")\n",
    "ax.zaxis.set_ticklabels([])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac7fc69d",
   "metadata": {},
   "source": [
    "### Question 5 : pouvez-vous afficher les 2 premières composantes principales ? Et comparer avec le diagramme 2D obtenu en sélectionnant les deux premières caractéristiques de X ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff6f4879",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plot2d(X_reduced[:,:2],y); plot2d(X[:,:2],y,labels[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f593d152",
   "metadata": {},
   "source": [
    "### Question 6: remarquez-vous ?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a64abf8",
   "metadata": {},
   "source": [
    "### Exercice à la maison: ACP et au-delà *en grande dimension*\n",
    "https://scikit-learn.org/stable/auto_examples/decomposition/plot_faces_decomposition.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada10311",
   "metadata": {},
   "source": [
    "### Pratique à la maison: approche non-linéaire avec tSNE sur IRIS\n",
    "https://github.com/olekscode/Examples-PCA-tSNE/blob/master/Python/Visualizing%20Iris%20Dataset%20using%20PCA%20and%20t-SNE.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "181718c7",
   "metadata": {},
   "source": [
    "# Partie 2 : partitionnement\n",
    "https://scikit-learn.org/stable/auto_examples/cluster/plot_cluster_iris.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aeed849f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "\n",
    "np.random.seed(5)\n",
    "\n",
    "estimators = [\n",
    "    (\"k_means_iris_8\", KMeans(n_clusters=8)),\n",
    "    (\"k_means_iris_3\", KMeans(n_clusters=3)),\n",
    "    (\"k_means_iris_bad_init\", KMeans(n_clusters=3, n_init=1, init=\"random\")),\n",
    "]\n",
    "\n",
    "fignum = 1\n",
    "titles = [\"8 clusters\", \"3 clusters\", \"3 clusters, bad initialization\"]\n",
    "for name, est in estimators:\n",
    "    fig = plt.figure(fignum, figsize=(4, 3))\n",
    "    ax = fig.add_subplot(111, projection=\"3d\", elev=48, azim=134)\n",
    "    ax.set_position([0, 0, 0.95, 1])\n",
    "    est.fit(X)\n",
    "    labels = est.labels_\n",
    "\n",
    "    ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=labels.astype(float), edgecolor=\"k\")\n",
    "\n",
    "    ax.w_xaxis.set_ticklabels([])\n",
    "    ax.w_yaxis.set_ticklabels([])\n",
    "    ax.w_zaxis.set_ticklabels([])\n",
    "    ax.set_xlabel(\"Petal width\")\n",
    "    ax.set_ylabel(\"Sepal length\")\n",
    "    ax.set_zlabel(\"Petal length\")\n",
    "    ax.set_title(titles[fignum - 1])\n",
    "    ax.dist = 12\n",
    "    fignum = fignum + 1\n",
    "\n",
    "# Plot the ground truth\n",
    "fig = plt.figure(fignum, figsize=(4, 3))\n",
    "ax = fig.add_subplot(111, projection=\"3d\", elev=48, azim=134)\n",
    "ax.set_position([0, 0, 0.95, 1])\n",
    "\n",
    "for name, label in [(\"Setosa\", 0), (\"Versicolour\", 1), (\"Virginica\", 2)]:\n",
    "    ax.text3D(\n",
    "        X[y == label, 3].mean(),\n",
    "        X[y == label, 0].mean(),\n",
    "        X[y == label, 2].mean() + 2,\n",
    "        name,\n",
    "        horizontalalignment=\"center\",\n",
    "        bbox=dict(alpha=0.2, edgecolor=\"w\", facecolor=\"w\"),\n",
    "    )\n",
    "# Reorder the labels to have colors matching the cluster results\n",
    "y = np.choose(y, [1, 2, 0]).astype(float)\n",
    "ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=y, edgecolor=\"k\")\n",
    "\n",
    "ax.w_xaxis.set_ticklabels([])\n",
    "ax.w_yaxis.set_ticklabels([])\n",
    "ax.w_zaxis.set_ticklabels([])\n",
    "ax.set_xlabel(\"Petal width\")\n",
    "ax.set_ylabel(\"Sepal length\")\n",
    "ax.set_zlabel(\"Petal length\")\n",
    "ax.set_title(\"Ground Truth\")\n",
    "ax.dist = 12\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c183d096",
   "metadata": {},
   "source": [
    "## Pratique à la maison: cas où les k-moyennes ne fonctionnent pas ou mal\n",
    "https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_assumptions.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
