Popular DB
- OLTP vs OLAP
OLTP (OnLine Transactional Processing) systems provide data to data warehouses. OLAP (OnLine Analytical Processing) systems help to analyze it, like periodically long-running batch jobs. A detailed comparison is here.
-
double write
-
how to do query optimization?
Aurora
write up by Gupta
Goals (for commercial, proprietary db)
1) reliability 2) high-performance 3) cost efficiency 4) scale
Amazon Aurora is a relational database service for OLTP workloads offered as part of Amazon Web Services (AWS). In this paper, we describe the architecture of Aurora and the design considerations leading to that architecture. We believe the central constraint in high throughput data processing has moved from compute and storage to the network. Aurora brings a novel architecture to the relational database to address this constraint, most notably by pushing redo processing to a multi-tenant scaleout storage service, purpose-built for Aurora. We describe how doing so not only reduces network traffic, but also allows for fast crash recovery, failovers to replicas without loss of data, and fault-tolerant, self-healing storage. We then describe how Aurora achieves consensus on durable state across numerous storage nodes using an efficient asynchronous scheme, avoiding expensive and chatty recovery protocols ...
However, amazon was designed in 2012 to 2013, at that time the network bandwidth is ~10GBs, now there are RDMA which support 25+Gbs. Amazon's RDS is based on EC2, where the storage is EBS, which is some cloud storage. EC2 connects with EBS via ethernet network.
Redo log as a innodb
6 replica??? gossip rather than paxos??
[1] Amazon Aurora 深度探索 - 腾讯技术工程的文章 - 知乎, https://zhuanlan.zhihu.com/p/33603518 [2]