Had same issue, you need to keep input sources open all the time when mixing the videos, what I mean that don't start input source in time frame of video mixing, secondly 24 different videos even for your system from my opinion is too much, because you have only 16 threads on 8 cores on cpu, that mean when you mixing the videos including live input the system use 24 tasks across 16 threads plus 1-2 tasks for live input, so for live input you have at least one core, so for mixing videos you remains with 7 cores
I have WO with 5930k cpu which have 6 cores 12 thread and msi 7970 graphics with 6 outputs and I was able to mix without lag 14 videos with 2 live sdi inputs, the clips were encoded WMV 15mbps