Adding Metrics section to capabilities in understanding domain #1068
Conversation
microsoft-github-policy-service
bot
added
the
Needs: Review 👀
microsoft-github-policy-service
bot
added
the
Tool: FinOps guide
| @@ -108,6 +108,21 @@ At this point, you have a data pipeline and are ingesting data into a central da | |||
|
|
|||
| <br> | |||
|
|
|||
| ## Data Ingestion Metrics | |||
There was a problem hiding this comment.
Let's use the same headers across all files so we can link to them generically. Also note we should use sentence casing rather than title casing to align with the Microsoft Style Guide.
| ## Data Ingestion Metrics | |
| ## KPIs and metrics |
| @@ -108,6 +108,21 @@ At this point, you have a data pipeline and are ingesting data into a central da | |||
|
|
|||
| <br> | |||
|
|
|||
| ## Data Ingestion Metrics | |||
|
|
|||
| | **Category** | **Definition** | **KPI** | | |||
There was a problem hiding this comment.
Update all of the categories to be sentence cased to align to the Microsoft Style Guide.
| @@ -108,6 +108,21 @@ At this point, you have a data pipeline and are ingesting data into a central da | |||
|
|
|||
| <br> | |||
|
|
|||
| ## Data Ingestion Metrics | |||
|
|
|||
| | **Category** | **Definition** | **KPI** | | |||
There was a problem hiding this comment.
Each KPI should include a formula. We may not be able to format this as a table.
|
|
||
| | **Category** | **Definition** | **KPI** | | ||
| |----------|-----------|-----| | ||
| | Data Completeness | Measures the extent to which all required data fields are present in the dataset and tracks the overall data completeness trend over a specified period.| Percentage of data fields that are complete and the overall data completeness over time. | |
There was a problem hiding this comment.
How feasible is it to measure this? I'm not pushing back. It sounds like the right thing to do, but do they have a way to actually measure it? How would we calculate this for them? Should we outline any potential challenges they may have in collecting this to give them a heads up? I'd hate for someone to take this list and say, "let's go track all these" and then realize there's no way to do it.
| @@ -108,6 +108,21 @@ At this point, you have a data pipeline and are ingesting data into a central da | |||
|
|
|||
| <br> | |||
|
|
|||
| ## Data Ingestion Metrics | |||
|
|
|||
| | **Category** | **Definition** | **KPI** | | |||
There was a problem hiding this comment.
Can you add each one of these into the backlog for adding to Power BI?
| |Data Ingestion Frequency | Measures how often data is ingested into the system. | Number of data ingestion events per unit of time (daily, weekly, monthly, quarterly, annually). | | ||
| | Volume of Data Ingested | Measures the total volume of data ingested into the repository. | Total volume of data ingested into the repository. | | ||
| | Growth Rate | Measure the rate at which the volume of data ingested is increasing over time. | Percentage increase of total data volume in repository per unit of time. | | ||
| | Ingestion Latency | Measures the average time taken for data to be ingested into the repository and tracks the trend of this latency over a specified period. | Mean time of data ingestion latency and the latency trend over a specified period. | |
There was a problem hiding this comment.
I like this one. A few thoughts:
- Do we need to call out that latency may differ by dataset?
- Do you intend to use "mean" time? Not average or percentile? All have merits, so just confirming.
- This can likely be split into multiple KPIs.
- Is latency trend a KPI or a visualization of a KPI over time? Not sure if visualizations need to be called out here unless we need to speak to the value of the visual. I'm open to either approach. Just thinking out loud to keep this simple. If we do keep it, it's probably a separate KPI that might be better if we can quantify a single number for it. Not sure 🤔
| | Volume of Data Ingested | Measures the total volume of data ingested into the repository. | Total volume of data ingested into the repository. | | ||
| | Growth Rate | Measure the rate at which the volume of data ingested is increasing over time. | Percentage increase of total data volume in repository per unit of time. | | ||
| | Ingestion Latency | Measures the average time taken for data to be ingested into the repository and tracks the trend of this latency over a specified period. | Mean time of data ingestion latency and the latency trend over a specified period. | | ||
| | Historical Data Availability | Measures the lookback period of data that is ingested and available for analysis. | Span of historical data ingested. | |
There was a problem hiding this comment.
This name needs some work, but I do like it. I've thought about this one as well. We need to know what data is missing so we can backfill it. Should this be bound to months with complete data over the retention/reporting period?
| | Volume of Data Ingested | Measures the total volume of data ingested into the repository. | Total volume of data ingested into the repository. | | ||
| | Growth Rate | Measure the rate at which the volume of data ingested is increasing over time. | Percentage increase of total data volume in repository per unit of time. | | ||
| | Ingestion Latency | Measures the average time taken for data to be ingested into the repository and tracks the trend of this latency over a specified period. | Mean time of data ingestion latency and the latency trend over a specified period. | | ||
| | Historical Data Availability | Measures the lookback period of data that is ingested and available for analysis. | Span of historical data ingested. | |
There was a problem hiding this comment.
This brings up a question about how much people are using historical data. We should probably talk about the cost of each month of data compared to the usage of that data. If people aren't using it, then that's wasted money. That will also help them quantify the value of storing the historical data.
| @@ -108,6 +108,21 @@ At this point, you have a data pipeline and are ingesting data into a central da | |||
|
|
|||
| <br> | |||
|
|
|||
| ## Data Ingestion Metrics | |||
|
|
|||
| | **Category** | **Definition** | **KPI** | | |||
There was a problem hiding this comment.
Can you think about the cost and carbon impact of each one of these? It may not apply everywhere. Anything that comes back to something that is metered, like data size or compute time.
| | Growth Rate | Measure the rate at which the volume of data ingested is increasing over time. | Percentage increase of total data volume in repository per unit of time. | | ||
| | Ingestion Latency | Measures the average time taken for data to be ingested into the repository and tracks the trend of this latency over a specified period. | Mean time of data ingestion latency and the latency trend over a specified period. | | ||
| | Historical Data Availability | Measures the lookback period of data that is ingested and available for analysis. | Span of historical data ingested. | | ||
| | Investigation Time to Resolution | Measures the time taken to investigate and resolve data quality or availability issues and tracks the trend of this resolution time over a specified period. | Mean time to investigate and resolve data quality or availability issues, and the trend over time. | |
There was a problem hiding this comment.
Similar comments about trends on this one. It's very interesting. This warrants its own backlog to think thru whether we have the right guidance to support it.
microsoft-github-policy-service
bot
added
Needs: Attention 👋
microsoft-github-policy-service
bot
added
Needs: Review 👀
KevDLR
changed the title
KevDLR
changed the title
@microsoft-github-policy-service agree company="Microsoft" |
flanakin
added
Needs: Attention 👋
🛠️ Description
Adding a Metrics/ KPI section to the capability to provide guidance on the Metric lens of the FinOps assessment.
Fixes
N/A
📋 Checklist
🔬 How did you test this change?
🙋♀️ Do any of the following that apply?
📑 Did you update
docs/changelog.md?📖 Did you update documentation?