blog

DeNAのエンジニアが考えていることや、担当しているサービスについて情報発信しています

2025.08.20 技術記事

What's New in Aurora DSQL? (Summary + Deep Dive into Internals) [DeNA Infra SRE]

by Hiroyuki Nishizaki

#infrastructure #database #aws #aurora #re:Invent #foundationdb

*This article is the English translation of a post originally published in Japanese on March 28, 2025 . Please be aware that the content reflects the time when Aurora DSQL was in its preview phase and not yet Generally Available (GA). Of particular note for this article, a blog post introducing “Crossbar” , a component of Aurora DSQL, was published on May 27, 2025. When evaluating Aurora DSQL, please ensure you always refer to the latest official information.

Hello, this is Hiro from Group 4 of the IT Platform Department. My primary role is to manage the infrastructure for mobile games that are released worldwide.

This is the third and final article in this series. In this concluding post, I will recap the previous articles and then explain how Aurora DSQL’s design is innovative from a database technology perspective.

Part 1 Recap: A Distributed SQL Database

In the first article, we compared Aurora DSQL with its predecessors like traditional Aurora. We introduced DSQL as a distributed SQL1 database and, in the context of DeNA’s own database operational challenges, highlighted its key benefits:

  • Sharding is No Longer Necessary
  • Your Data is Always Up-to-date
  • Higher Availability With Zero Infrastructure Management

Part 2 Recap: Advanced Component Separation + Optimistic Concurrency Control

In the second article, I compared DSQL with Google Cloud Spanner. We found that DSQL, also a distributed SQL database, features an even more fine-grained separation of transaction-related components than Spanner. Coupled with its adoption of optimistic concurrency control, this provides two main advantages:

  • Lower latency
    • Especially in multi-region deployments or when updating data across multiple shards.
  • Independent Scaling of Performance Aspects
    • Though the concrete user benefits are still awaiting future announcements.

Part 3 [This Article]: Technical Design and Innovations of DSQL

When considering Aurora DSQL, I believe the first question that comes to mind is, “So, what can I actually do with it?” This was the very question that motivated my own research, which is why this series so far has focused on comparing its features with existing products from an architectural standpoint.

However, a closer look reveals a different picture. Many of the technologies within Aurora DSQL are not entirely new; they have existed before. While this might also imply that DSQL’s most innovative aspects are yet to be announced, this time I will explore what we know today, tracing how Aurora DSQL utilizes existing database technologies and what benefits it gains as a result.

Please note that references to existing technologies are largely based on my own knowledge and speculation. While I will provide sources, I cannot guarantee their accuracy or completeness. Database technology is a vast and complex field, so please consider this an effort to deepen understanding of a new service based on my personal analysis.

Component Separation + Optimistic Concurrency Control

Benefit: Low-latency transactions and flexible scalability.

The component separation and transaction management model of Aurora DSQL, which we introduced in the last article, is remarkably similar to an Apple OSS database, FoundationDB(which also uses optimistic concurrency control). While FoundationDB is a Key-Value Store at its core, it provides strong transactional consistency and allows for flexible scaling of all performance aspects, including writes. By using additional software called “Record Layer,” it can also handle relational data, allowing it to be used much like a distributed SQL database2.

Comparison with FoundationDB

First, let’s revisit the diagram for an Aurora DSQL read and write transaction.

Aurora DSQL transaction

Now, here is the diagram for FoundationDB3.

FoundationDB transaction

  • Sequencer
    • This component issues globally unique versions for the entire FoundationDB cluster.
    • There is only one Sequencer in a FoundationDB cluster.
  • Proxy
    • A Proxy obtains a transaction version from the Sequencer.
    • For a write transaction, it checks with the Resolver for conflicts.
    • If there are no conflicts, it sends the write operation to the LogServer.
  • Resolver
    • A Resolver checks if a write operation issued by the Proxy would violate data consistency (i.e., if there are conflicting transactions).
  • LogServer
    • A LogServer receives the data to be written from the Proxy, makes it durable, and asynchronously propagates the changes to the StorageServers.
  • StorageServers
    • They store the actual data.

Comparing the two, we can see the following similarities:

  • Unit of Component (Function) Separation:
    • Receiving user requests, reading data, and performing computations
      • Client / Query Processor
    • Issuing transaction version information (timestamps)
      • Sequencer / Query Processor
    • Detecting conflicting transactions
      • Resolver / Adjudicator
    • Persisting data to be written (Write-Ahead-Log)
      • LogServer / Journal
    • Distributed storage
      • StorageServer / Storage
  • Optimistic Concurrency Control
    • Conflict detection is performed at commit time by a Resolver / Adjudicator, and the transaction is aborted if any conflict is found.

Each of FoundationDB’s components (except for unique ones like the Sequencer) can also be scaled independently, which is why its read and write performance can scale independently, just like DSQL. By adopting a similar component architecture and optimistic concurrency control, Aurora DSQL became a highly scalable, low-latency distributed SQL.

Interestingly, while I mentioned in the first article that DSQL has a 5-minute transaction timeout limit, FoundationDB also has a transaction timeout, and it’s only 5 seconds4. This suggests that while DSQL inherits a limitation from this design philosophy, it has been made comparatively more user-friendly.

Globally Synchronized Timestamps

Benefit: Multi-region deployments

Looking at the diagrams, you’ll notice that FoundationDB’s Sequencer component is absent in Aurora DSQL. This is because FoundationDB, unlike DSQL, does not assume the availability of globally synchronized (low-error) timestamps. Instead, a Client must obtain a corresponding version number from the dedicated component called Sequencer via a Proxy. The Sequencer is the single source for unique versioning in the entire cluster.

If one were to attempt a multi-region, active-active configuration with FoundationDB like DSQL’s, this Sequencer would become a bottleneck, creating a significant latency trade-off.

FoundationDB trade-off

Aurora DSQL, on the other hand, enables its Query Processors to obtain accurate (low-error) timestamps locally, eliminating the need to fetch version information from a specific, central location. This architectural change is what realizes the key feature we’ve discussed: “in a multi-region Aurora DSQL setup, no cross-region communication occurs until commit time.” By introducing globally synchronized timestamps, Aurora DSQL can support multi-region deployments while maintaining low latency. It also gains the benefits of even faster performance by removing the latency of querying a Sequencer, and it eliminates a scaling bottleneck by distributing the version-issuing process.

Aurora DSQL sameregion

Globally synchronized timestamps are also used by Spanner, which achieves this via its TrueTime API. As mentioned in the first article, DSQL accomplishes this using the Amazon Time Sync Service 5.

Pushdown to Storage

Benefit: High-performance processing of relational data

Aurora DSQL’s Storage layer now returns partially processed row data to the Query Processor instead of raw page data. This means that data operations like selections, joins, and projections are performed in the Storage layer. Processing data closer to where it lives reduces latency. It’s explicitly stated that this improvement allows DSQL to achieve higher performance than simply using a Key-Value Store (like FoundationDB) for relational data. It also enables more rapid scaling of the compute layer because the Query Processors no longer need to cache page data, removing the need for synchronization between them6.

Pushing down data processing to the storages to improve efficiency is a common technique in distributed SQL. Spanner, too, evolved from a traditional key-value store-like storage implementation to one better suited for relational data, while also changing its query processing to a model where operations like selections and projections are performed in parallel on the servers holding the data, with the results aggregated in a separate layer7.

At the same time, this improvement is quintessentially “Amazon Aurora”. The original Amazon Aurora storage was developed to optimize for the cloud environment by reducing network usage, which is often a bottleneck in large-scale distributed environments8. This storage pushdown is a logical extension of that philosophy, further reducing data transfer from storage to compute. It’s a fascinating and excellent example of how to handle relational data in a distributed environment.

Various Internal AWS Technologies

Benefit: Improved development efficiency and enhanced DSQL’s own performance and reliability.

Aurora DSQL reuses a variety of internal AWS technologies9.

The Query Processors use Firecracker MicroVM containing the customized Postgres engine, the same technology developed for and used by AWS Lambda and AWS Fargate. It also leverages other virtualization technologies, such as Caspian (a system for dynamically scaling virtual server resources, also used in Aurora Serverless v2) and the snapshotting technology used in Lambda SnapStart .

Furthermore, Journal itself is the distributed transaction log that was already in use within AWS. The development of DSQL was made more efficient by formal methods and automated reasoning tool set employed internally at AWS. The reuse of these proven, battle-tested technologies has resulted in a high-performance, reliable system.

Summary: Design as a Fast, Global Distributed SQL is the Innovation.

The combination of these technologies and their benefits gives rise to Aurora DSQL: a “fast, highly scalable, global distributed SQL database.”

  • Component Separation + Optimistic Concurrency
    • Resulted in a low-latency distributed SQL database with flexible scalability.
  • Use of Globally Accurate Timestamps
    • Enabled multi-region deployments.
    • Eliminated the latency and scaling bottleneck associated with version fetching.
  • Pushdown to Storage
    • Enabled high-performance processing of relational data.
  • Various Internal AWS Technologies
    • Improved development efficiency and enhanced DSQL’s own performance and reliability.

While we can expect more information, including new technologies, to be revealed in the future, it is very fascinating and interesting to see how Aurora DSQL is an amalgamation of various technologies from both inside and outside AWS.

Viewed this way, Aurora DSQL can be seen as a distributed SQL database optimized for using relational data across multiple regions with the minimum possible latency from each region. Indeed, one article of Marc’s Blog, a primary source for this series, states that there was strong customer demand for such a database, and DSQL was designed precisely to meet that need10.

Final Thoughts: How should we view Aurora DSQL?

I’ve been researching databases for a while, and DSQL’s announcement came at the perfect time, so I decided to write this series. It was my first time writing a technical blog series and was quite challenging, but I’m now feeling relieved to have completed it.

On a personal note, I find it incredibly interesting that AWS chose to build its flagship distributed SQL product on a foundation of optimistic concurrency control, which presupposes small, low-conflict transactions. While the conclusion of this series is that this was done to enable low-latency multi-region use, there is still much we don’t know about this preview product. I’m very curious about its future development direction: will the transaction timeout be relaxed? Are pessimistic locking use cases completely delegated to traditional Aurora?

How we understand a new product and guess its evolution is not just an interesting exercise; it can directly impact our technology choices. For a company like DeNA that uses databases at a large scale and for the long term, these choices can determine the fate of projects like our game titles. The expertise gained is reused within our teams, influencing future technology adoption and strategies. When using AWS services, we are dependent on their availability, cost, and future roadmap. Therefore, understanding how AWS positions a service, how they plan to develop it, and whether that aligns with DeNA’s use cases is critical information for our decision-making.

Although Aurora DSQL is still a preview feature, I hope I’ve shown that it’s possible to analyze and understand new technologies by comparing them with existing ones. Regardless of the system type or scale, and even in a cloud-first environment, it is always crucial to understand the technologies we use and to choose them wisely. Here in the IT Platform Division, we will continue to investigate and validate a wide range of technologies.


  1. “NewSQL” is a similar term, but in this series, I have used “distributed SQL” to align with DSQL’s name and the official announcement . Both terms describe products that combine the features of traditional relational databases with the ability to scale write performance flexibly. However, my understanding is that “NewSQL” often emphasizes a comparison with NoSQL systems, while “distributed SQL” tends to refer to a narrower group of products and focuses on the benefits derived from distributed processing. These terms are fluid and change with context. For example, a paper published in 2016 by the researcher who coined the term NewSQL lists Amazon Aurora as one of the most notable examples of NewSQL. The same paper excludes FoundationDB from NewSQL, classifying it as a Key-Value Store, yet the FoundationDB paper itself claims NewSQL properties. Since the goal of this series is to deepen the understanding of DSQL itself, I have focused on explaining its specific functions and design rather than debating terminology. ↩︎

  2. FoundationDB development began in 2009. It was acquired by Apple in 2015 and open-sourced in 2018. It has a long track record of use in production environments at Apple, Snowflake, and VMWare. Although it’s a Key-Value Store, it’s designed to store various data types like relational data and JSON by placing a “layer” software between it and the application. However, it does not support ANSI SQL statements.
    https://www.foundationdb.org/files/fdb-paper.pdf  ↩︎

  3. For clarity, the diagram is based on a 2021 paper that shows similarities to Aurora DSQL. The latest architecture has some changes, such as separate proxies for reads and writes, and a “Master” component handling the Sequencer’s function. This explanation is limited to the Data Plane, which is directly involved in transactions. ↩︎

  4. https://apple.github.io/foundationdb/known-limitations.html#long-transactions  ↩︎

  5. https://brooker.co.za/blog/2024/12/03/aurora-dsql.html
    https://brooker.co.za/blog/2024/12/04/inside-dsql.html  ↩︎

  6. https://brooker.co.za/blog/2024/12/04/inside-dsql.html  ↩︎

  7. https://static.googleusercontent.com/media/research.google.com/ja//pubs/archive/46103.pdf  ↩︎

  8. The original Aurora was designed to avoid replicating entire data pages. Instead, the writer instance sends only redo logs to the Aurora storage, which then handles log-based replication internally.
    https://www.amazon.science/publications/amazon-aurora-design-considerations-for-high-throughput-cloud-native-relational-databases  ↩︎

  9. https://brooker.co.za/blog/2024/12/03/aurora-dsql.html
    https://brooker.co.za/blog/2024/12/04/inside-dsql.html  ↩︎

  10. https://brooker.co.za/blog/2024/12/03/aurora-dsql.html  ↩︎

最後まで読んでいただき、ありがとうございます!
この記事をシェアしていただける方はこちらからお願いします。

recruit

DeNAでは、失敗を恐れず常に挑戦し続けるエンジニアを募集しています。