Confirm
358
edits
Changes
→Things To Think About
* Needs a detailed and careful plan for how we'll bring up new DBs for existing shards, how we'll move dshards between DBs, and how we'll split shards if that becomes necessary. All very doable, just fiddly.
<mmayo> [21:22:58] rfkelly|away: telliott: rnewman: will reply to PICL storage thread soon, but if I forget
the TLDR; version is: we should plan for a caching tier not in AWS
<mmayo> [21:23:20] mechanism TBD
<mmayo> [21:23:39] but basically keep the hot transactions on high-spindle DB servers in a datacenter.
<mmayo> [21:24:05] since god-awful I/O rates are still really expensive and shitty in EC2.
<mmayo> might be as simple as detecting hot "shards", might be more sophisticated.
<mmayo> but it would be very nice to have some form of hierarchical storage management as far of the design.
<mmayo> I was doing some Cassandra testing instead of sleeping the other night, and even the biggest EC2 instances
can only do about 1/2 the IOPS of a bare metal, lesser machine.
