01-Getting_Started_with_Azure_ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started with Azure Machine Learning\n",
    "\n",
    "Azure Machine Learning (*Azure ML*) is a cloud-based service for creating and managing machine learning solutions. It's designed to help data scientists leverage their existing data processing and model development skills and frameworks, and help them scale their workloads to the cloud. The Azure ML SDK for Python provides classes you can use to work with Azure ML in your Azure subscription.\n",
    "\n",
    "## Before You Start\n",
    "\n",
    "1. Complete the steps in [Lab 1 - Getting Started with Azure Machine Learning](./labdocs/Lab01.md) to create an Azure Machine Learning workspace and a compute instance with the contents of this repo.\n",
    "2. Open this notebook in the compute instance and run it there.\n",
    "\n",
    "## Check the Azure ML SDK Version\n",
    "\n",
    "Let's start by importing the **azureml-core** package and checking the version of the SDK that is installed. Click the cell below and then use the **&#9658; Run** button on the toolbar to run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "print(\"Ready to use Azure ML\", azureml.core.VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to Your Workspace\n",
    "\n",
    "All experiments and associated resources are managed within you Azure ML workspace. You can connect to an existing workspace, or create a new one using the Azure ML SDK.\n",
    "\n",
    "In most cases, you should store the workspace configuration in a JSON configuration file. This makes it easier to reconnect without needing to remember details like your Azure subscription ID. You can download the JSON configuration file from the blade for your workspace in the Azure portal, but if you're using a Compute Instance within your workspace, the configuration file has alreday been downloaded to the root folder.\n",
    "\n",
    "The code below uses the configuration file to connect to your workspace. The first time you run it in a notebook session, you'll be prompted to sign into Azure by clicking the https://microsoft.com/devicelogin link,  entering an automatically generated code, and signing into Azure. After you have successfully signed in, you can close the browser tab that was opened, return to this notebook, and wait for the sign-in process to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "print(ws.name, \"loaded\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run an Experiment\n",
    "\n",
    "One of the most fundamentals tasks that data scientists need to perform is to create and run experiments that process and analyze data. In this exercise, you'll learn how to use an Azure ML *experiment* to run Python code and record values extracted from data. In this case, you'll use a simple dataset that contains details of patients that have been tested for diabetes. You'll run an experiment to explore the data, extracting statistics, visualizations, and data samples. Most of the code you'll use is fairly generic Python, such as you might run in any data exploration process. However, with the addition of a few lines, the code uses an Azure ML *experiment* to log details of the run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "\n",
    "# Create an Azure ML experiment in your workspace\n",
    "experiment = Experiment(workspace = ws, name = \"diabetes-experiment\")\n",
    "\n",
    "# Start logging data from the experiment\n",
    "run = experiment.start_logging()\n",
    "print(\"Starting experiment:\", experiment.name)\n",
    "\n",
    "# load the data from a local file\n",
    "data = pd.read_csv('data/diabetes.csv')\n",
    "\n",
    "# Count the rows and log the result\n",
    "row_count = (len(data))\n",
    "run.log('observations', row_count)\n",
    "print('Analyzing {} rows of data'.format(row_count))\n",
    "\n",
    "# Plot and log the count of diabetic vs non-diabetic patients\n",
    "diabetic_counts = data['Diabetic'].value_counts()\n",
    "fig = plt.figure(figsize=(6,6))\n",
    "ax = fig.gca()    \n",
    "diabetic_counts.plot.bar(ax = ax) \n",
    "ax.set_title('Patients with Diabetes') \n",
    "ax.set_xlabel('Diagnosis') \n",
    "ax.set_ylabel('Patients')\n",
    "plt.show()\n",
    "run.log_image(name = 'label distribution', plot = fig)\n",
    "\n",
    "# log distinct pregnancy counts\n",
    "pregnancies = data.Pregnancies.unique()\n",
    "run.log_list('pregnancy categories', pregnancies)\n",
    "\n",
    "# Log summary statistics for numeric columns\n",
    "med_columns = ['PlasmaGlucose', 'DiastolicBloodPressure', 'TricepsThickness', 'SerumInsulin', 'BMI']\n",
    "summary_stats = data[med_columns].describe().to_dict()\n",
    "for col in summary_stats:\n",
    "    keys = list(summary_stats[col].keys())\n",
    "    values = list(summary_stats[col].values())\n",
    "    for index in range(len(keys)):\n",
    "        run.log_row(col, stat = keys[index], value = values[index])\n",
    "        \n",
    "# Save a sample of the data and upload it to the experiment output\n",
    "data.sample(100).to_csv('sample.csv', index=False, header=True)\n",
    "run.upload_file(name = 'outputs/sample.csv', path_or_stream = './sample.csv')\n",
    "\n",
    "# Complete the run\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View Experiment Results\n",
    "\n",
    "After the experiment has been finished, you can use the **run** object to get information about the run and its outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "# Get run details\n",
    "details = run.get_details()\n",
    "print(details)\n",
    "\n",
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "print(json.dumps(metrics, indent=2))\n",
    "\n",
    "# Get output files\n",
    "files = run.get_file_names()\n",
    "print(json.dumps(files, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Jupyter Notebooks, you can use the **RunDetails** widget to get a better visualization of the run details, while the experiment is running or after it has finished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.widgets import RunDetails\n",
    "\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the **RunDetails** widget includes a link to view the run in Azure Machine Learning studio. Click this to open a new browser tab with the run details (you can also just open [Azure Machine Learning studio](https://ml.azure.com) and find the run on the **Experiments** page). When viewing the run in Azure Machine Learning studio, note the following:\n",
    "\n",
    "- The **Details** tab contains the general properties of the experiment run.\n",
    "- The **Metrics** tab enables you to select logged metrics and view them as tables or charts.\n",
    "- The **Images** tab enables you to select and view any images or plots that were logged in the experiment (in this case, the *Label Distribution* plot)\n",
    "- The **Child Runs** tab lists any child runs (in this experiment there are none).\n",
    "- The **Outputs + Logs** tab shows the output or log files generated by the experiment.\n",
    "- The **Snapshot** tab contains all files in the folder where the experiment code was run (in this case, everything in the same folder as this notebook).\n",
    "- The **Explanations** tab is used to show model explanations generated by the experiment (in this case, there are none).\n",
    "- The **Fairness** tab is used to visualize predictive performance disparities that help you evaluate the fairness of machine learning models (in this case, there are none).\n",
    "\n",
    "## Run an Experiment Script\n",
    "\n",
    "In the previous example, you ran an experiment inline in this notebook. A more flexible solution is to create a separate script for the experiment, and store it in a folder along with any other files it needs, and then use Azure ML to run the experiment based on the script in the folder.\n",
    "\n",
    "First, let's create a folder for the experiment files, and copy the data into it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "folder_name = 'diabetes-experiment-files'\n",
    "experiment_folder = './' + folder_name\n",
    "os.makedirs(folder_name, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(folder_name, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll create a Python script containing the code for our experiment, and save it in the experiment folder.\n",
    "\n",
    "> **Note**: running the following cell just *creates* the script file - it doesn't run it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $folder_name/diabetes_experiment.py\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes dataset\n",
    "data = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Count the rows and log the result\n",
    "row_count = (len(data))\n",
    "run.log('observations', row_count)\n",
    "print('Analyzing {} rows of data'.format(row_count))\n",
    "\n",
    "# Count and log the label counts\n",
    "diabetic_counts = data['Diabetic'].value_counts()\n",
    "print(diabetic_counts)\n",
    "for k, v in diabetic_counts.items():\n",
    "    run.log('Label:' + str(k), v)\n",
    "      \n",
    "# Save a sample of the data in the outputs folder (which gets uploaded automatically)\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "data.sample(100).to_csv(\"outputs/sample.csv\", index=False, header=True)\n",
    "\n",
    "# Complete the run\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code is a simplified version of the inline code used before. However, note the following:\n",
    "- It uses the `Run.get_context()` method to retrieve the experiment run context when the script is run.\n",
    "- It loads the diabetes data from the folder where the script is located.\n",
    "- It creates a folder named **outputs** and writes the sample file to it - this folder is automatically uploaded to the experiment run\n",
    "\n",
    "Now you're almost ready to run the experiment. To run the script, you must create a **ScriptRunConfig** that identifies the Python script file to be run in the experiment, and then run an experiment based on it.\n",
    "\n",
    "> **Note**: The ScriptRunConfig also determines the compute target and Python environment. If you don't specify these, a default environment is created automatically on the local compute where the code is being run (in this case, where this notebook is being run).\n",
    "\n",
    "The following cell configures and submits the script-based experiment.\n",
    "\n",
    "> **Note**: This will take a little longer to run the first time, as the environment must be created. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "from azureml.core import Experiment, ScriptRunConfig\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "\n",
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=experiment_folder, \n",
    "                      script='diabetes_experiment.py') \n",
    "\n",
    "# submit the experiment\n",
    "experiment = Experiment(workspace = ws, name = 'diabetes-experiment')\n",
    "run = experiment.submit(config=script_config)\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, you can use the widget or the link to the experiment in [Azure Machine Learning studio](https://ml.azure.com) to view the outputs generated by the experiment, and you can also write code to retrieve the metrics and files it generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View Experiment Run History\n",
    "\n",
    "Now that you've run experiments multiple times, you can view the history in [Azure Machine Learning studio](https://ml.azure.com) and explore each logged run. Or you can retrieve an experiment by name from the workspace and iterate through its runs using the SDK:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment, Run\n",
    "\n",
    "diabetes_experiment = ws.experiments['diabetes-experiment']\n",
    "for logged_run in diabetes_experiment.get_runs():\n",
    "    print('Run ID:', logged_run.id)\n",
    "    metrics = logged_run.get_metrics()\n",
    "    for key in metrics.keys():\n",
    "        print('-', key, metrics.get(key))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use MLflow\n",
    "\n",
    "MLflow is an open source platform for managing machine learning processes. It's commonly (but not exclusively) used in Databricks environments to coordinate experiments and track metrics. In Azure Machine Learning experiments, you can use MLflow to track metrics instead of the native log functionality if you desire.\n",
    "\n",
    "### Use MLflow with an Inline Experiment\n",
    "\n",
    "To use MLflow to track metrics for an inline experiment, you must set the MLflow *tracking URI* to the workspace where the experiment is being run. This enables you to use **mlflow** tracking methods to log data to the experiment run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment\n",
    "import pandas as pd\n",
    "import mlflow\n",
    "\n",
    "# Set the MLflow tracking URI to the workspace\n",
    "mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())\n",
    "\n",
    "# Create an Azure ML experiment in your workspace\n",
    "experiment = Experiment(workspace=ws, name='diabetes-mlflow-experiment')\n",
    "mlflow.set_experiment(experiment.name)\n",
    "\n",
    "# start the MLflow experiment\n",
    "with mlflow.start_run():\n",
    "    \n",
    "    print(\"Starting experiment:\", experiment.name)\n",
    "    \n",
    "    # Load data\n",
    "    data = pd.read_csv('data/diabetes.csv')\n",
    "\n",
    "    # Count the rows and log the result\n",
    "    row_count = (len(data))\n",
    "    print('observations:', row_count)\n",
    "    mlflow.log_metric('observations', row_count)\n",
    "    \n",
    "# Get a link to the experiment in Azure ML studio        \n",
    "experiment_url = experiment.get_portal_url()\n",
    "print('See details at', experiment_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After running the code above, you can use the link that is displayed to view the experiment in Azure Machine Learning studio. Then select the latest run of tghe experiment and view it **Metrics** tab to see the logged metric.\n",
    "\n",
    "### Use MLflow in an Experiment Script\n",
    "\n",
    "You can also use MLflow to track metrics in an experiment script.\n",
    "\n",
    "Run the following two cells to create a folder and a script for an experiment that uses MLflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "folder_name = 'mlflow-experiment-files'\n",
    "experiment_folder = './' + folder_name\n",
    "os.makedirs(folder_name, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(folder_name, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $folder_name/mlflow_diabetes.py\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import mlflow\n",
    "\n",
    "# start the MLflow experiment\n",
    "with mlflow.start_run():\n",
    "       \n",
    "    # Load data\n",
    "    data = pd.read_csv('diabetes.csv')\n",
    "\n",
    "    # Count the rows and log the result\n",
    "    row_count = (len(data))\n",
    "    print('observations:', row_count)\n",
    "    mlflow.log_metric('observations', row_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you use MLflow tracking in an Azure ML experiment script, the MLflow tracking URI is set automatically when you start the experiment run. However, the environment in which the script is to be run must include the required **mlflow** packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from azureml.core import Experiment, ScriptRunConfig, Environment\n",
    "from azureml.core.conda_dependencies import CondaDependencies\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "\n",
    "# Create a Python environment for the experiment\n",
    "mlflow_env = Environment(\"mlflow-env\")\n",
    "\n",
    "# Ensure the required packages are installed\n",
    "packages = CondaDependencies.create(pip_packages=['mlflow', 'azureml-mlflow'])\n",
    "mlflow_env.python.conda_dependencies = packages\n",
    "\n",
    "# Create a script config\n",
    "script_mlflow = ScriptRunConfig(source_directory=experiment_folder,\n",
    "                      script='mlflow_diabetes.py',\n",
    "                      environment=mlflow_env) \n",
    "\n",
    "# submit the experiment\n",
    "experiment = Experiment(workspace = ws, name = 'diabetes-mlflow-experiment')\n",
    "run = experiment.submit(config=script_mlflow)\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, you can get the logged metrics from the experiment run when it's finished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you've seen how to use the Azure ML SDK to view the resources in your workspace and run experiments. \n",
    "\n",
    "### Learn More\n",
    "\n",
    "- For more details about the SDK, see the [Azure ML SDK documentation](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py).\n",
    "- To find out more about running experiments, see [Start, monitor, and cancel training runs in Python](https://docs.microsoft.com/azure/machine-learning/how-to-manage-runs) in the Azure ML documentation.\n",
    "- For details of how to log metrics in a run, see [Monitor Azure ML experiment runs and metrics](https://docs.microsoft.com/azure/machine-learning/how-to-track-experiments).\n",
    "- For more information about integrating Azure ML experiments with MLflow, see [Track model metrics and deploy ML models with MLflow and Azure Machine Learning](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-mlflow).\n",
    "\n",
    "## Clean Up\n",
    "\n",
    "On the **File** menu, click **Close and Halt** to close this notebook. Then close all Jupyter tabs in your browser and **stop** your compute instance to minimize costs."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}