Exposing Prometheus metrics from an ASP.NET Core application

by Joel

Prometheus

"Prometheus is an open-source systems monitoring and alerting toolkit" and has becode the defacto standard for exposing metrics from your applications, specially when running on Kubernetes.

In combination with a dashboarding tool like Grafana it's a great way of consolidating metrics from different applications and visualizing them in dashboards for monitoring and alerting purposes.

(By default) Prometheus will scrape an http endpoint on your application, and expect to find metrics defined in a specific format.

prometheus-net for ASP.NET Core

Let's see how we can start exposing some default metrics in this Prometheus format from our ASP.NET Core applications.

We will create a simple ASP.NET Core 5 application, and use prometheus-net for the metrics. You can view the examples on GitHub.

Create a new ASP.NET Core web API:
dotnet new webapi -n PrometheusNetDemo

It's just a default project that exposes some fake data on https://localhost:5001/weatherforecast

Exposing default ASP.NET Core data

Since we're using ASP.NET Core, we need to use the prometheus-net.AspNetCore nuget package:
dotnet add package prometheus-net.AspNetCore --version 4.1.1

All you need to do now is to add the following to your Startup.cs:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    ...
    app.UseRouting();
    app.UseHttpMetrics(); // Make sure this is called after app.UseRouting()

    app.UseEndpoints(endpoints =>
    {
        ...
        endpoints.MapMetrics();
    });
}

That's really all there is to it. Now, you're exposing some default ASP.NET Core metrics in a Prometheus format on https://localhost:5001/metrics. It will look something like this and include both information about memory usage, garbage collection, requests, etc:

# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 26026536960
# HELP process_private_memory_bytes Process private memory size
# TYPE process_private_memory_bytes gauge
process_private_memory_bytes 0
# HELP process_num_threads Total number of threads
# TYPE process_num_threads gauge
process_num_threads 24
# HELP process_open_handles Number of open handles
# TYPE process_open_handles gauge
process_open_handles 0
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 1.3847572
# HELP dotnet_collection_count_total GC collection count
# TYPE dotnet_collection_count_total counter
dotnet_collection_count_total{generation="1"} 0
dotnet_collection_count_total{generation="0"} 0
dotnet_collection_count_total{generation="2"} 0
# HELP dotnet_total_memory_bytes Total known allocated memory
# TYPE dotnet_total_memory_bytes gauge
dotnet_total_memory_bytes 3958584
# HELP process_working_set_bytes Process working set
# TYPE process_working_set_bytes gauge
process_working_set_bytes 60047360
# HELP http_requests_received_total Provides the count of HTTP requests that have been processed by the ASP.NET Core pipeline.
# TYPE http_requests_received_total counter
http_requests_received_total{code="404",method="GET",controller="",action=""} 2
http_requests_received_total{code="200",method="GET",controller="WeatherForecast",action="Get"} 1
# HELP http_requests_in_progress The number of requests currently in progress in the ASP.NET Core pipeline. One series without controller/action label values counts all in-progress requests, with separate series existing for each controller-action pair.
# TYPE http_requests_in_progress gauge
http_requests_in_progress{method="GET",controller="",action=""} 0
http_requests_in_progress{method="GET",controller="WeatherForecast",action="Get"} 0
# HELP http_request_duration_seconds The duration of HTTP requests processed by an ASP.NET Core application.
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_sum{code="404",method="GET",controller="",action=""} 0.0035634
http_request_duration_seconds_count{code="404",method="GET",controller="",action=""} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.001"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.002"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.004"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.008"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.016"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.032"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.064"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.128"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.256"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.512"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="1.024"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="2.048"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="4.096"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="8.192"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="16.384"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="32.768"} 2
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="+Inf"} 2
http_request_duration_seconds_sum{code="200",method="GET",controller="WeatherForecast",action="Get"} 0.0633071
http_request_duration_seconds_count{code="200",method="GET",controller="WeatherForecast",action="Get"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.001"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.002"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.004"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.008"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.016"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.032"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.064"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.128"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.256"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="0.512"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="1.024"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="2.048"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="4.096"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="8.192"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="16.384"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="32.768"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="WeatherForecast",action="Get",le="+Inf"} 1
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1618082481.927617

Exposing metrics on a different port

By default, the metrics will be exposed on the same port as the rest of your API. There are many reasons why you might not want to do that. Maybe this is a public facing API and you do not want your metrics out in the open. So how do we solve that?

Instead of using endpoints.MapMetrics();, we should do the following:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    app.UseMetricServer(9102);

    ...
    
    app.UseRouting();
    app.UseHttpMetrics();

Exposing custom metrics

What we've done so far is only to expose some default metrics from our ASP.NET Core application. What might be useful is to create some custom metrics and expose them. That's another topic that I won't cover here, but you can read more about it.