We have observed some performance gains when using 10M direct buffers instead of 1M direct buffers. It is my hypothesis that this gain is entirely due to the amount of IO that can be driven by the write cache thread that actually writes on the disk and the network replication channel. My thought is that larger write cache buffers in effect allow us to schedule more write IOs for a fixed dirty list threshold. For example, if we have a dirty list threshold of 5 and 1MB buffers, then we can schedule up to 5MB of writes. However if we have the same dirty list threshold and 10M buffers, then we can schedule up to 50MB of writes.
If this is true, then we should adjust the default dirty list threshold. This is much safer than playing around with the direct buffer pool size since HA requires the same size buffers on all nodes.