HikariCP keeps connecting to the old writer after Blue/Green Switchover
After an Aurora Blue/Green Switchover, a HikariCP-based application kept connecting to the old writer (now demoted to reader). All 398 successful queries stayed on the old IP — it never switched to the Green environment's new writer.
According to AWS documentation, all connections to both environments are dropped during switchover. However, when HikariCP creates new connections afterward, if the DNS cache still returns the old writer's IP, it reconnects to the old environment (now demoted to reader). Aurora cluster endpoints have a DNS TTL of 60 seconds, so there is a risk of staying connected to the old environment during this window. Read queries succeed, but writes fail with read-only transaction errors.
This behavior is identical on both PostgreSQL and MySQL — it's engine-agnostic. It's a DNS cache and connection pool reconnection timing issue. Depending on DNS propagation timing, new connections may reach the new writer instead.
Mitigations:
- Use the AWS JDBC Driver BG plugin (IP-based routing, bypasses DNS entirely)
- Set a shorter
maxLifetimein HikariCP to periodically discard old connections - Use
connectionTestQueryfor connection validation (but this alone doesn't guarantee switching to the new writer)
See Aurora Blue/Green in Practice Part 2 for detailed test results.
