After setting up Prometheus for metrics and Grafana for dashboards, there was still one piece missing. Logs.
I had been shipping logs to different places over the years. A database table early on, then Azure Application Insights, then a managed Elasticsearch cluster. Each time, the setup worked but came with trade-offs I kept accepting without questioning. Elasticsearch in particular is powerful, but it is also expensive and operationally heavy for what most applications actually need from logs.
Loki is the answer I wish I had found earlier.
What Makes Loki Different
Most log aggregation systems index the full content of every log line. Elasticsearch does this. That indexing is what makes searches fast, but it also makes ingestion expensive and storage large. You end up paying for full-text search on logs that you search maybe once a month.
Loki takes a different approach. It only indexes the labels attached to a log stream, not the log content itself. The actual log text is stored compressed and only read when you query it. This makes Loki dramatically cheaper to run and simpler to operate. The trade-off is that filtering by log content requires scanning, not index lookups. In practice, for most workloads, this is completely fine.
The mental model is the same as Prometheus. Labels identify a stream, content is the payload. If you already understand how Prometheus labels work, Loki immediately makes sense.
Setting It Up with .NET
In a .NET application, Serilog is the logging library I use. Wiring it to Loki takes one package and a few lines of configuration.
dotnet add package Serilog.Sinks.Grafana.LokiLog.Logger = new LoggerConfiguration()
.WriteTo.GrafanaLoki("http://localhost:3100",
labels: new[] {
new LokiLabel { Key = "app", Value = "my-api" },
new LokiLabel { Key = "env", Value = "production" }
})
.CreateLogger();Those labels are how Loki identifies the stream. Every log line from this application will be tagged with app=my-api and env=production. You can then query all logs for that app, or filter by environment, without touching the log content at all.
Keep labels low-cardinality. Do not use request IDs or user IDs as labels. Those belong in the log line itself, not as stream identifiers. High-cardinality labels create too many streams and defeat Loki's architecture.
LogQL Is Straightforward
Loki's query language is called LogQL. It has two modes: log queries that return log lines, and metric queries that turn log data into numbers (useful for counting errors over time, for example).
A basic query looks like this:
{app="my-api", env="production"} |= "Exception"The curly braces select the stream by label. The pipe filters log lines containing the word "Exception". That is most of what you need day to day.
You can also parse structured log fields out of the line:
{app="my-api"} | json | status_code >= 500If your application logs JSON (which it should), Loki can parse the fields on the fly and let you filter on them without any pre-indexing. This is genuinely useful and something Elasticsearch would require index mappings for.
The Grafana Connection
In Grafana, you add Loki as a data source alongside Prometheus. From that point, you can build panels that combine both. A dashboard showing error rate from Prometheus metrics and the actual error log lines from Loki on the same screen is something I use regularly. You see the spike in the metric and immediately jump to the logs for that time window without switching tools.
Grafana also has a feature called Explore that is basically a raw query interface for both Prometheus and Loki. I use it constantly when debugging. You switch between metrics and logs in the same view, correlate timestamps, and move from "something went wrong at 14:32" to "here is the exact stack trace" in a few clicks.
Compared to What I Was Using Before
Application Insights does logs too. It is fine. But it is Azure-only, it gets expensive at volume, and the query language (Kusto) is something I always had to look up rather than actually knowing.
The managed Elasticsearch I used on one project cost more per month than the entire rest of the infrastructure. It was fast and the full-text search was powerful, but most of that power was never used. We searched logs by service name, time range, and error level. Loki handles all of that easily at a fraction of the cost.
Running Loki yourself is simple. It is a single binary or a Docker container. The storage can be local disk or object storage like S3 or Azure Blob. For a small team's production workload, you can run it for almost nothing.
The Full Picture
With Prometheus, Loki, and Grafana together, you have the full observability stack. Metrics tell you something is wrong. Logs tell you what exactly happened. Grafana connects both in one place.
None of it is tied to a specific cloud provider. All three are open source. Your dashboards, your alert rules, your log queries are all portable. That matters more than it seems when you are setting things up, and it matters a lot when something changes later.
If you have already moved to Prometheus and Grafana, adding Loki is the natural next step. It closes the loop.
