Machine Learning with Scikit-Learn - The Cancer Dataset - 9 - Logistic Regression 2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "## <center>Supervised Learning</center>\n",
    "\n",
    "- making inferences from labeled data.\n",
    "\n",
    "#### 1. Classification (categorical data)\n",
    "- binary classification (tumor: benign, malignant)\n",
    "- multiclass classification (books: maths, physics, stats, psychology, etc.)\n",
    "- example algorithms: KNN, Linear Models, Decision Trees, SVMs, etc.\n",
    "\n",
    "#### 2. Regression (continuous data)\n",
    "- predicting income, price of stock, age, and other continous data \n",
    "- example algorithms: KNN, Linear Models, Decision Trees, SVMs, etc.\n",
    "___\n",
    "\n",
    "Linear models (LinReg, LogReg, Lasso, Ridged, etc) - make predictions according to a linear function of the input features. <br>\n",
    "Many ML algorithms (including those specified above) can be used for both classification and regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using LogisticRegression on the cancer dataset. Inspired by Muller and Guido ML book: (https://www.amazon.com/dp/1449369413/)\n",
    "\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "cancer = load_breast_cancer()\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target, stratify=cancer.target, random_state=42)\n",
    "\n",
    "log_reg = LogisticRegression()\n",
    "log_reg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on the training subset: 0.953\n",
      "Accuracy on the test subset: 0.958\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy on the training subset: {:.3f}'.format(log_reg.score(X_train, y_train)))\n",
    "print('Accuracy on the test subset: {:.3f}'.format(log_reg.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Regularization**:\n",
    "\n",
    "- prevention of overfitting - (according to Muller and Guido ML book)\n",
    "- L1 - assumes only a few features are important\n",
    "- L2 - does not assume only a few features are important - used by default in scikit-learn LogisticRegression\n",
    "               \n",
    "**'C'**:\n",
    "\n",
    "- parameter to control the strength of regularization\n",
    "- lower C => log_reg adjusts to the majority of data points.\n",
    "- higher C => correct classification of each data point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on the training subset: 0.972\n",
      "Accuracy on the test subset: 0.965\n"
     ]
    }
   ],
   "source": [
    "log_reg100 = LogisticRegression(C=100)\n",
    "log_reg100.fit(X_train, y_train)\n",
    "print('Accuracy on the training subset: {:.3f}'.format(log_reg100.score(X_train, y_train)))\n",
    "print('Accuracy on the test subset: {:.3f}'.format(log_reg100.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on the training subset: 0.934\n",
      "Accuracy on the test subset: 0.930\n"
     ]
    }
   ],
   "source": [
    "log_reg001 = LogisticRegression(C=0.01)\n",
    "log_reg001.fit(X_train, y_train)\n",
    "print('Accuracy on the training subset: {:.3f}'.format(log_reg001.score(X_train, y_train)))\n",
    "print('Accuracy on the test subset: {:.3f}'.format(log_reg001.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}