Nav modernizes its data logging and frees engineers to innovate

To support thousands of weekly production changes, the Norwegian Labor and Welfare Administration (Nav) replaced its centralized Elasticsearch logging stack with Aiven for OpenSearch. The move eliminated infrastructure burden, ensured reliable high-volume log processing and integrated seamlessly with Aiven for Apache Kafka. The result: zero production disruptions, greater developer productivity and optimized data-logging practices, all from a scalable, usage-based cost structure aligned with Nav’s cloud-first, open-source strategy.

Building a cloud-native welfare platform for Norway

In Norway, the promise of essential, timely and efficient welfare payments is delivered by the Norwegian Labor and Welfare Administration (Nav). It is Norway’s largest government agency, managing roughly one-third of the country's annual state budget, and touching almost every resident during their lifetime. Pensions, parental leave, disability benefits and the broader social safety net for which the Scandinavian countries are known all fall under Nav’s remit.

Nav operates at scale, with around 1,000 employees in its Oslo offices, and a further 21,000 working in smaller municipalities nationwide. For years, the technology infrastructure supporting that scale was run by internal architects and project managers, while external consultants delivered solutions, which often resulted in sluggish and expensive project structures.

However, over the past decade, Nav has made a deliberate break from that model, adopting cloud and cloud-native technologies, primarily through Google Cloud, and enabling developers with better tools to build, deploy and manage applications in a DevOps model. It has also shifted towards having more in-house developers, who are organized into small, autonomous product teams.

Logging at massive scale and under pressure

The agency adopted Apache Kafka® early in its modernization program, but as it moved deeper into cloud infrastructure, it acknowledged that running its own Kafka clusters was not where it delivered value. Nav turned to Aiven for Apache Kafka to manage its mission-critical Kafka clusters. Nav subsequently adopted Aiven for OpenSearch to run small but meaningful workloads, including the search functionality behind nav.no, as well as internal caseworker systems, and Norway's official job marketplace. It also later adopted Aiven for Valkey for caching.

The next challenge was to address data logging.

The agency pays out legally mandated and financially critical welfare benefits to millions of people, and a delayed or incorrect payment can cause real harm, especially for vulnerable populations. Logging enables Nav to verify that its systems are performing lawfully, accurately and in line with the agency’s own quality standards. Even though the agency is moving toward a broader OpenTelemetry ecosystem, with traces and metrics, logs remain critically important.

At Nav's scale, logging alone produces around 400 GB of data every day and requires 60 TB of data to be retained. The agency deploys approximately 4,000 production changes per week, and logs play a key role in maintaining that pace. “When logging works, it's invisible,” says Hans Kristian Flaatten, Platform Engineer at Nav. “When it doesn't work, the consequences spread fast. Developers no longer have visibility across the system. We can’t safely release changes or troubleshoot incidents.” 

Nav was operating centralized, on-premises logging built on Elasticsearch. But the arrangement was becoming increasingly untenable, as licensing costs increased and operational demands intensified. Performance wasn’t at a level that an organization of Nav’s size and scope actually needed and, crucially, maintaining the infrastructure required expertise that had little to do with Nav's core mission.

“We had already migrated Kafka to Aiven, which is truly mission-critical. If Kafka goes down, welfare services stop,” says Flaatten. “But like our original Kafka clusters, operating and maintaining our on-premises Elasticsearch logging stack had become a burden. Aiven was already hosting some of our most critical data services, including Kafka, so expanding that partnership to OpenSearch felt like a natural evolution.”

Migrating to Aiven for OpenSearch

The agency was already familiar with Aiven for OpenSearch, which made it the obvious solution to replace Elasticsearch for the centralized logging and observability stack, which was Nav's largest cluster by data volume.

The team began with a proof of concept to validate OpenSearch's capabilities against the Elasticsearch environment it was leaving behind and worked through feature differences before committing to a full cutover.

Once the POC was complete, the migration itself was relatively straightforward. Because Nav retains logs for 30 to 90 days, there was no need to move historical data. The team stood up the OpenSearch cluster on GCP with Aiven, began routing logs to both systems in parallel, and waited for the retention windows to fill before performing performance and data consistency checks.

The whole migration, from decision to completion, ran for roughly four months. Throughout, Nav and Aiven maintained weekly check-ins alongside continuous collaboration. “We needed some custom configuration work to meet our specific performance needs,” says Flaatten, “Aiven’s responsiveness, particularly from the product team, stood out to our engineers throughout the migration. That partnership had a meaningful impact on the project’s success.”

Following the migration of its logging infrastructure to Aiven for OpenSearch, Nav relies on Aiven for Kafka to manage inbound event streaming, enabling high-throughput pipelines and real-time log analytics. With Kafka, OpenSearch and Valkey all running through Aiven, Nav has consolidated critical data services under a unified, cloud-native operating model — giving product teams a reliable and scalable foundation within its managed cloud architecture.

From firefighting to forward momentum

Since completing the migration, Nav has experienced no production issues with its logging infrastructure. More importantly, the shift has freed up engineering capacity. The engineers who once focused on keeping the logging service running still oversee it, but instead of reacting to operational incidents, they now work proactively on improvements and optimization. But the impact goes beyond operational resilience, it has also streamlined how Nav’s teams integrate and manage infrastructure within their internal platform.

“Aiven’s API-first approach plays a key role in improving developer productivity,” says Flaatten. “Our teams shouldn’t have to manage multiple consoles. Because everything is API-driven, we can integrate Aiven’s services directly into our own platform interface.”

With Aiven for OpenSearch in place, Flaatten’s colleagues are also much better able to help product teams use logs more effectively, which raises the quality of structured logging across the agency. “Aiven for OpenSearch optimizes how we retain and query data,” says Flaatten. “We’ve moved away from operational toil to building capacity.”

Although reducing spending wasn't the primary driver of the project, in moving to an operational model that is better suited to modern cloud infrastructure, it has shifted the cost model from capital-intensive infrastructure to usage-based operational expenditure. Having stepped away from proprietary systems that create dependency and limit flexibility, logging is now aligned with Nav's broader cloud migration and open-source-first strategies.

“By working with Aiven, we now have a consumption-based cost structure that scales with actual use,” says Flaatten. “As our teams continue to optimize logging practices and retention policies, we can expect more efficiencies in the future.”

Unifying expertise around OpenSearch

With the logging migration complete, Nav is building more internal expertise around OpenSearch, closing off the split knowledge base that came from running Elasticsearch and OpenSearch in parallel. A unified internal technology group now supports the platform, with teams actively upskilling through OpenSearch conferences. “We expect that centralized expertise will translate into faster, more consistent support for product teams across the organization,” says Flaatten.

Nav is continuing to work with Aiven to refine how it handles Kafka topic provisioning through APIs, as well as tightening permission management and improving monitoring integrations and internal dashboards.

“We’ve been fortunate to work closely with Aiven’s engineers and project managers throughout this process. Rapid responsiveness and strong collaboration have defined this project,” says Flaatten. “We’re proud of what we’ve achieved, because strengthening our technical foundation ultimately strengthens our ability to serve society.”