Merge pull request MicrosoftDocs#13542 from spelluru/adfmonitormanage…

…0518

reviewed, tested, and changed

articles/data-factory/data-factory-monitor-manage-app.md

@@ -13,7 +13,7 @@ ms.workload: data-services
 ms.tgt_pltfrm: na
 ms.devlang: na
 ms.topic: article
-ms.date: 02/21/2017
+ms.date: 05/18/2017
 ms.author: spelluru
 
 ---
@@ -24,14 +24,16 @@ ms.author: spelluru
 >
 >
 
-This article describes how to use the Monitoring and Management app to monitor, manage, and debug your Azure Data Factory pipelines--and create alerts to get notified on failures. You can also watch the following video to learn about using the Monitoring and Management app.
+This article describes how to use the Monitoring and Management app to monitor, manage, and debug your Data Factory pipelines. It also provides information on how to create alerts to get notified on failures. You can get started with using the application by watching the following video:
+
+> [!NOTE]
+> The user interface shown in the video may not exactly match what you see in the portal. It's slightly older, but concepts remain the same. 
 
 > [!VIDEO https://channel9.msdn.com/Shows/Azure-Friday/Azure-Data-Factory-Monitoring-and-Managing-Big-Data-Piplines/player]
 >
->
 
-## Open the Monitoring and Management app
-To open the Monitor and Management app, click the **Monitor & Manage** tile on the **Data Factory** blade for your data factory.
+## Launch the Monitoring and Management app
+To launch the Monitor and Management app, click the **Monitor & Manage** tile on the **Data Factory** blade for your data factory.
 
 ![Monitoring tile on the Data Factory home page](./media/data-factory-monitor-manage-app/MonitoringAppTile.png)
 
@@ -42,7 +44,13 @@ You should see the Monitoring and Management app open in a separate window.
 > [!NOTE]
 > If you see that the web browser is stuck at "Authorizing...", clear the **Block third-party cookies and site data** check box--or keep it selected, create an exception for **login.microsoftonline.com**, and then try to open the app again.
 
-If you don't see activity windows in the list at the bottom, click the **Refresh** button on the toolbar to refresh the list. In addition, set the right values for the **Start time** and **End time** filters.  
+
+In the Activity Windows list in the middle pane, you see an activity window for each run of an activity. For example, if you have the activity scheduled to run hourly for five hours, you see five activity windows associated with five data slices. If you don't see activity windows in the list at the bottom, do the following:
+
+- Update the **start time** and **end time** filters at the top to match the start and end times of your pipeline, and then click the **Apply** button.  
+- The Activity Windows list is not automatically refreshed. Click the **Refresh** button on the toolbar in the **Activity Windows** list.  
+
+If you don't have a Data Factory application to test these steps with, do the tutorial: [copy data from Blob Storage to SQL Database using Data Factory](data-factory-copy-data-from-azure-blob-storage-to-sql-database.md).
 
 ## Understand the Monitoring and Management app
 There are three tabs on the left: **Resource Explorer**, **Monitoring Views**, and **Alerts**. The first tab (**Resource Explorer**) is selected by default.
@@ -51,9 +59,9 @@ There are three tabs on the left: **Resource Explorer**, **Monitoring Views**, a
 You see the following:
 
 * The Resource Explorer **tree view** in the left pane.
-* The **Diagram View** at the top.
+* The **Diagram View** at the top in the middle pane.
 * The **Activity Windows** list at the bottom in the middle pane.
-* The **Properties** and **Activity Window Explorer** tabs in the right pane.
+* The **Properties**, **Activity Window Explorer**, and **Script** tabs in the right pane.
 
 In Resource Explorer, you see all resources (pipelines, datasets, linked services) in the data factory in a tree view. When you select an object in Resource Explorer:
 
@@ -77,23 +85,31 @@ When the pipeline is enabled (not in a paused state), it's shown with a green li
 
 ![Pipeline running](./media/data-factory-monitor-manage-app/PipelineRunning.png)
 
-There are three command bar buttons for the pipeline in the Diagram View. You can use the second button to pause the pipeline. Pausing doesn't terminate the currently running activities and lets them proceed to completion. The third button pauses the pipeline and terminates its existing executing activities. The first button resumes the pipeline. When your pipeline is paused, the color of the pipeline changes to yellow:
+You can pause, resume, or terminate a pipeline by selecting it in the diagram view and using the buttons on the command bar.
 
-![Pause/resume on tile](./media/data-factory-monitor-manage-app/SuspendResumeOnTile.png)
+![Pause/resume on the command bar](./media/data-factory-monitor-manage-app/SuspendResumeOnCommandBar.png)
+
+There are three command bar buttons for the pipeline in the Diagram View. You can use the second button to pause the pipeline. Pausing doesn't terminate the currently running activities and lets them proceed to completion. The third button pauses the pipeline and terminates its existing executing activities. The first button resumes the pipeline. When your pipeline is paused, the color of the pipeline changes. For example, a paused pipeline looks like in the following image: 
 
-You can multiselect two or more pipelines by using the Ctrl key. You can use the command bar buttons to pause/resume multiple pipelines at a time.
+![Pipeline paused](./media/data-factory-monitor-manage-app/PipelinePaused.png)
 
-![Pause/resume on the command bar](./media/data-factory-monitor-manage-app/SuspendResumeOnCommandBar.png)
+You can multi-select two or more pipelines by using the Ctrl key. You can use the command bar buttons to pause/resume multiple pipelines at a time.
+
+You can also right-click a pipeline and select options to suspend, resume, or terminate a pipeline. 
 
-You can see all the activities in the pipeline by right-clicking the pipeline tile, and then clicking **Open pipeline**.
+![Context menu for pipeline](./media/data-factory-monitor-manage-app/right-click-menu-for-pipeline.png)
+
+Click the **Open pipeline** option to see all the activities in the pipeline. 
 
 ![Open pipeline menu](./media/data-factory-monitor-manage-app/OpenPipelineMenu.png)
 
-In the opened pipeline view, you see all activities in the pipeline. In this example, there is only one activity: Copy Activity. To go back to the previous view, click the data factory name in the breadcrumb menu at the top.
+In the opened pipeline view, you see all activities in the pipeline. In this example, there is only one activity: Copy Activity. 
 
 ![Opened pipeline](./media/data-factory-monitor-manage-app/OpenedPipeline.png)
 
-In the pipeline view, when you click an output dataset or when you move your mouse over the output dataset, you see the Activity Windows pop-up window for that dataset.
+To go back to the previous view, click the data factory name in the breadcrumb menu at the top.
+
+In the pipeline view, when you select an output dataset or when you move your mouse over the output dataset, you see the Activity Windows pop-up window for that dataset.
 
 ![Activity Windows pop-up window](./media/data-factory-monitor-manage-app/ActivityWindowsPopup.png)
 

articles/data-factory/data-factory-monitor-manage-pipelines.md

@@ -13,7 +13,7 @@ ms.workload: data-services
 ms.tgt_pltfrm: na
 ms.devlang: na
 ms.topic: article
-ms.date: 02/21/2017
+ms.date: 05/18/2017
 ms.author: spelluru
 
 ---
@@ -23,15 +23,11 @@ ms.author: spelluru
 > * [Using Monitoring and Management app](data-factory-monitor-manage-app.md)
 
 
-Azure Data Factory provides a reliable and complete view of your storage, processing, and data movement services. The service provides you a monitoring dashboard that you can use to:
+> [!IMPORTANT]
+> The monitoring & management application provides a better support for monitoring and managing your data pipelines, and troubleshooting any issues. For details about using the application, see [monitor and manage Data Factory pipelines by using the Monitoring and Management app](data-factory-monitor-manage-app.md). 
 
-* Quickly assess end-to-end data pipeline health.
-* Identify issues, and take corrective action if needed.
-* Track data lineage.
-* Track relationships between your data across any of your sources.
-* View full historical accounting of job execution, system health, and dependencies.
 
-This article describes how to monitor, manage, and debug your pipelines. It also provides information on how to create alerts and get notified about failures.
+This article describes how to monitor, manage, and debug your pipelines by using Azure portal and PowerShell. The article also provides information on how to create alerts and get notified about failures.
 
 ## Understand pipelines and activity states
 By using the Azure portal, you can:
@@ -40,15 +36,13 @@ By using the Azure portal, you can:
 * View activities in a pipeline.
 * View input and output datasets.
 
-This section also describes how a slice transitions from one state to another state.   
+This section also describes how a dataset slice transitions from one state to another state.   
 
 ### Navigate to your data factory
 1. Sign in to the [Azure portal](https://portal.azure.com).
 2. Click **Data factories** on the menu on the left. If you don't see it, click **More services >**, and then click **Data factories** under the **INTELLIGENCE + ANALYTICS** category.
 
    ![Browse all > Data factories](./media/data-factory-monitor-manage-pipelines/browseall-data-factories.png)
-
-   You should see all the data factories on the **Data factories** blade.
 3. On the **Data factories** blade, select the data factory that you're interested in.
 
     ![Select data factory](./media/data-factory-monitor-manage-pipelines/select-data-factory.png)
@@ -58,13 +52,11 @@ This section also describes how a slice transitions from one state to another st
    ![Data factory blade](./media/data-factory-monitor-manage-pipelines/data-factory-blade.png)
 
 #### Diagram view of your data factory
-The **Diagram** view of a data factory provides a single pane of glass to monitor and manage the data factory and its assets.
-
-To see the **Diagram** view of your data factory, click **Diagram** on the home page for the data factory.
+The **Diagram** view of a data factory provides a single pane of glass to monitor and manage the data factory and its assets. To see the **Diagram** view of your data factory, click **Diagram** on the home page for the data factory.
 
 ![Diagram view](./media/data-factory-monitor-manage-pipelines/diagram-view.png)
 
-You can zoom in, zoom out, zoom to fit, zoom to 100%, lock the layout of the diagram, and automatically position pipelines and tables. You can also see the data lineage information (that is, show upstream and downstream items of selected items).
+You can zoom in, zoom out, zoom to fit, zoom to 100%, lock the layout of the diagram, and automatically position pipelines and datasets. You can also see the data lineage information (that is, show upstream and downstream items of selected items).
 
 ### Activities inside a pipeline
 1. Right-click the pipeline, and then click **Open pipeline** to see all activities in the pipeline, along with input and output datasets for the activities. This feature is useful when your pipeline includes more than one activity and you want to understand the operational lineage of a single pipeline.
@@ -168,17 +160,13 @@ The slice starts in a **Waiting** state, waiting for preconditions to be met bef
 
 You can reset the slice to go back from the **Ready** or **Failed** state to the **Waiting** state. You can also mark the slice state to **Skip**, which prevents the activity from executing and not processing the slice.
 
-## Manage pipelines
-You can manage your pipelines by using Azure PowerShell. For example, you can pause and resume pipelines by running Azure PowerShell cmdlets.
-
-### Pause and resume pipelines
-You can pause/suspend pipelines by using the **Suspend-AzureRmDataFactoryPipeline** PowerShell cmdlet. This cmdlet is useful when you don’t want to run your pipelines until an issue is fixed.
+## Pause and resume pipelines
+You can manage your pipelines by using Azure PowerShell. For example, you can pause and resume pipelines by running Azure PowerShell cmdlets. 
 
-For example, in the following screenshot, an issue has been identified with the **PartitionProductsUsagePipeline** in the **productrecgamalbox1dev** data factory, and we want to suspend the pipeline.
+> [!NOTE] 
+> The diagram view does not support pausing and resuming pipelines. If you want to use an user interface, use the monitoring and managing application. For details about using the application, see [monitor and manage Data Factory pipelines by using the Monitoring and Management app](data-factory-monitor-manage-app.md) article. 
 
-![Pipeline to be suspended](./media/data-factory-monitor-manage-pipelines/pipeline-to-be-suspended.png)
-
-To suspend a pipeline, run the following PowerShell command:
+You can pause/suspend pipelines by using the **Suspend-AzureRmDataFactoryPipeline** PowerShell cmdlet. This cmdlet is useful when you don’t want to run your pipelines until an issue is fixed. 
 
 ```powershell
 Suspend-AzureRmDataFactoryPipeline [-ResourceGroupName] <String> [-DataFactoryName] <String> [-Name] <String>
@@ -189,7 +177,7 @@ For example:
 Suspend-AzureRmDataFactoryPipeline -ResourceGroupName ADF -DataFactoryName productrecgamalbox1dev -Name PartitionProductsUsagePipeline
 ```
 
-After the issue has been fixed with the **PartitionProductsUsagePipeline**, you can resume the suspended pipeline by running the following PowerShell command:
+After the issue has been fixed with the pipeline, you can resume the suspended pipeline by running the following PowerShell command:
 
 ```powershell
 Resume-AzureRmDataFactoryPipeline [-ResourceGroupName] <String> [-DataFactoryName] <String> [-Name] <String>
@@ -199,9 +187,13 @@ For example:
 ```powershell
 Resume-AzureRmDataFactoryPipeline -ResourceGroupName ADF -DataFactoryName productrecgamalbox1dev -Name PartitionProductsUsagePipeline
 ```
+
 ## Debug pipelines
 Azure Data Factory provides rich capabilities for you to debug and troubleshoot pipelines by using the Azure portal and Azure PowerShell.
 
+> [!NOTE} 
+> It is much easier to troubleshot errors using the Monitoring & Management App. For details about using the application, see [monitor and manage Data Factory pipelines by using the Monitoring and Management app](data-factory-monitor-manage-app.md) article. 
+
 ### Find errors in a pipeline
 If the activity run fails in a pipeline, the dataset that is produced by the pipeline is in an error state because of the failure. You can debug and troubleshoot errors in Azure Data Factory by using the following methods.
 
@@ -217,7 +209,7 @@ If the activity run fails in a pipeline, the dataset that is produced by the pip
    ![Activity run details blade with error](./media/data-factory-monitor-manage-pipelines/activity-run-details-with-error.png)     
 
 #### Use PowerShell to debug an error
-1. Start **Azure PowerShell**.
+1. Launch **PowerShell**.
 2. Run the **Get-AzureRmDataFactorySlice** command to see the slices and their statuses. You should see a slice with the status of **Failed**.        
 
 	```powershell   
@@ -229,7 +221,7 @@ If the activity run fails in a pipeline, the dataset that is produced by the pip
 	Get-AzureRmDataFactorySlice -ResourceGroupName ADF -DataFactoryName LogProcessingFactory -DatasetName EnrichedGameEventsTable -StartDateTime 2014-05-04 20:00:00
 	```
 
-   Replace **StartDateTime** with the the StartDateTime value that you specified for the Set-AzureRmDataFactoryPipelineActivePeriod.
+   Replace **StartDateTime** with start time of your pipeline. 
 3. Now, run the **Get-AzureRmDataFactoryRun** cmdlet to get details about the activity run for the slice.
 
 	```powershell   
@@ -275,12 +267,17 @@ If the activity run fails in a pipeline, the dataset that is produced by the pip
 	```
 
 ## Rerun failures in a pipeline
+
+> [!IMPORTANT]
+> It's easier to troubleshoot errors and rerun failed slices by using the Monitoring & Management App. For details about using the application, see [monitor and manage Data Factory pipelines by using the Monitoring and Management app](data-factory-monitor-manage-app.md). 
+
 ### Use the Azure portal
 After you troubleshoot and debug failures in a pipeline, you can rerun failures by navigating to the error slice and clicking the **Run** button on the command bar.
 
 ![Rerun a failed slice](./media/data-factory-monitor-manage-pipelines/rerun-slice.png)
 
 In case the slice has failed validation because of a policy failure (for example, if data isn't available), you can fix the failure and validate again by clicking the **Validate** button on the command bar.
+
 ![Fix errors and validate](./media/data-factory-monitor-manage-pipelines/fix-error-and-validate.png)
 
 ### Use Azure PowerShell