Network tweaks

[jochri]

I just want to quickly share some basic network tweaks.

When dealing with 10G and 40G networks there are settings in the network drivers which become very important for the file transfer speeds.

As the Intel X-550 series card is one of the most used cards I will explain with their settings. Most other cards use the same names for the settings:

Interrupt Moderation: OFF / The interrupt moderation prevents the CPU from getting the load from the network card. Means when to much traffic is incoming the network card will not give away load to the cpu but will simply "moderate" the incoming files to a level it can handle and also to save energy. While for many server applications this might be the right way of handling traffic especially WATCHOUT has no load on the cpu while recieving files. So you can use the resources to support the network card.

The same setting for interrupt moderation can be found in the driver under the performance options menu selection. Switch interrupt moderation: OFF

Send & recieve buffers: Set to the maximum your card can support.

RSS queues: Set to the maximum.

A good way of testing these settings is to use a simple windows network file transfer. Open up the task manager and see what happens with the cpu load with or without interrupt moderation. Multiple file transfers connected between 10 G / 40G to 10G / 40G pcs load the cpu to more then 75% on a new 8 core machine.

When handling large ammounts of data these settings can safe hours of transfer time. And while the Intel X550 is a good product also think about RDMA capable cards and manufacterers like Mellanox or ATTO when choosing the right network cards. The differences are bigger then one might think - so are the options of the network card settings which will also expand.

Didnt tested with many NDI channels so far. If anyone might share their test results in combination with input moderation settings that would be super cool.

Whatever you do keep testing!

By the way. Just finished a poduction in china where bad unshielded network cables were creating big problems. So use high quality cables for the connections! Otherwise they might create lots of problems.

[lewk]

This is awesome! Curious - can you share details of what the cheap unshielded network cables were doing?