clustering-DBSCAN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "75210344-c3a1-49ec-bb94-37132f4602af",
   "metadata": {},
   "source": [
    "# Кластеризация методом DBSCAN\n",
    "Используем реализацию [DBSCAN](https://scikit-learn.org/stable/modules/clustering.html#dbscan) в [scikit-learn](https://scikit-learn.org/stable/index.html) (метод [DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html#sklearn.cluster.DBSCAN))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac38bc12-a19c-4e85-9a9c-705152843100",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.cluster import DBSCAN\n",
    "from sklearn.metrics import silhouette_score, calinski_harabasz_score, davies_bouldin_score\n",
    "from sklearn.preprocessing import StandardScaler  # z-нормировка\n",
    "# from sklearn.preprocessing import MinMaxScaler, RobustScaler # другие способы нормировки\n",
    "\n",
    "# import seaborn as sns # 2D-визуализация\n",
    "import plotly.express as px # 3D-визуализация\n",
    "\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# Не показывать Warning\n",
    "import warnings\n",
    "warnings.simplefilter(action='ignore', category=Warning)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed7864d1-efeb-4bb6-9461-bc5038b2b6e6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('./datasets/Countries.csv')\n",
    "# Удалим нечисловые переменные\n",
    "X = df.drop(columns=['Страны'])\n",
    "X.index = df['Страны']\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a42f17a-5c99-447d-b293-6d13935eb76e",
   "metadata": {},
   "source": [
    "## Нормирование данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a99b367-5e52-4bf7-acfb-9ab9d0db4394",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Специфицируем метод нормировки\n",
    "scaler = StandardScaler(with_mean=True, with_std=True)\n",
    "# scaler = MinMaxScaler()\n",
    "# scaler = RobustScaler()\n",
    "\n",
    "X_norm = scaler.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a549985-b0e0-40dc-824d-aeef9c17842c",
   "metadata": {},
   "source": [
    "## Кластеризация\n",
    "\n",
    "*Важно*: выбросы имеют номер -1!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14ffa1c2-9700-4dd6-8059-bc826490458f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Специфицируем процедуру\n",
    "cluster = DBSCAN(eps=1, min_samples=4, metric='euclidean')\n",
    "# проводим кластеризацию\n",
    "cluster.fit(X_norm)\n",
    "\n",
    "print(cluster.labels_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8fcdc29-c66d-414a-8b04-a8d94d33ca6c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "n_clusters_ = len(set(cluster.labels_)) - (1 if -1 in cluster.labels_ else 0)\n",
    "n_noise_ = list(cluster.labels_).count(-1)\n",
    "\n",
    "print('Estimated number of clusters: %d' % n_clusters_)\n",
    "print('Estimated number of noise points: %d' % n_noise_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d06c6b44-9550-48e5-aed6-f72346123b30",
   "metadata": {},
   "source": [
    "## Визуализация\n",
    "используем библиотеки [seaborn](https://seaborn.pydata.org/index.html) и [plotly](https://plotly.com/python/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e70accc9-4d82-4a5a-a5f0-9339ddd44f02",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "X['Cluster'] = np.astype(cluster.labels_, str)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58e7a6d6",
   "metadata": {},
   "source": [
    "2D визуализация в seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca1b4d2f-0d31-42e9-b11a-5252284e7ad2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Визуализация в seaborn\n",
    "# sns.scatterplot(data=X, x='ИРЧП', y='Население', hue='Cluster')\n",
    "# sns.scatterplot(data=X, x='ИРЧП', y='ВВП д/н', hue='Cluster')\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca2d6eb1",
   "metadata": {},
   "source": [
    "2D визуализация в plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "caea9965",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(X, x='ИРЧП', y='Население', color='Cluster', hover_name=X.index)\n",
    "# fig.update_traces(marker_size=6)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c223791",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(X, x='ИРЧП', y='ВВП д/н', color='Cluster', hover_name=X.index)\n",
    "# fig.update_traces(marker_size=5)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2bbfffe",
   "metadata": {},
   "source": [
    "3D визуализация в plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6de2a96-078a-45d0-a6f1-5b1827902ef0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = px.scatter_3d(X, x='ИРЧП', y='ВВП д/н', z='Безработица (%)', color='Cluster', hover_name=X.index)\n",
    "fig.update_traces(marker_size=4)\n",
    "fig.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}