This tight interweaving of removes and adds described above is NOT efficient. The correct way to do this is to batch all removes together followed by batching all asserts together. This can be 1000s of times more efficient than applying the remove/assert one subject/predicate at a time.
I have assigned this issue to MikeP, who is working on a unit test for the problem. Once that is available, I will take the issue back to work on a fix. The problem appears to be related to large branching factors and retention queues used with the RWStore. Truth maintenance is performance against a TemporaryTripleStore, which is based by a TemporaryRawStore which is a kind of WORM. It appears that the large branching factor is driving up the memory demand. However, there is probably a memory leak which is preventing the timely finalization of the TemporaryTripleStores associated with the assert/retract pattern.
Another memory inefficiency is the buffer capacity for the statement buffer associated with truth maintenance. Normally a large buffer capacity is used for the SAIL (the default is 10k and people may configured values as high as 1M for higher throughput when loading data). When the same large buffer capacity is applied to truth maintenance, especially for such small deltas, there is significant memory waste in the StatementBuffer's internal data structures.