jochri
-
Posts
34 -
Joined
-
Last visited
Posts posted by jochri
-
-
You can use VMWARE V-SPHERE to run multiple VMs with GPU PCIe passthrough out of one server. Just useful for very large deployments...
0 -
i have build quite a lot of these systems.
These are typically enterprise plattforms which make certain things easier and somethings harder.
Go for Gen 4 PCIe - RTX A5000 should be the right card - fill every memory channel - set your bios correct - consider buy a server of the shelf or from me
Dual CPU is working good.
0 -
Switch off the interrupt moderation in your production PC card and your clients.
Maximize send and receive buffers. Intel cars are cool but if you want to reach maximum performance go for Melllanox or Atto cards!
Keep in mind also the max write speed of your media disks. M.2 is getting hot after 30 seconds. So go for u.2 or PCIE cards. PM1725a Intel P4510 or HGST....
0 -
I currently redesign the Wiki. Please find the Bios tweaks here:
- Sleep States: Talking about real time applications you should get rid of all settings which makes your CPU going to save energy. They are often called C- States and should be set so that your PC never sleeps. Sometimes you can also find a setting which is called "Block Sleep" (S3 State) So thats what you need to check and set to not stop your mediaserver from working at full power. Some mainboards have high performance modes. This is a good starting point. But dont just use auto tuning settings!
- Intel Speed Step: Activate or not activate? The answer is it depends! Older CPUs, I would say before 2017 needed a certain amount of time to go from idle to full power (60-80ms). This milliseconds can cause too much delay to your mediaserver system. So on older systems it is neccesary to deactivate this setting to always run the CPU at a certain level to avoid the delays. - Since Intel introduced the Speed Shift function this setting changed. In a short way explained. Speed Shift accelerates the speed at which the CPU can go to full power to just a few microseconds(1-3ms). So activate it if you are using new hardware that supports speedshift! The CPU should run at full speed without changing frequeny up and down. You can verify this easily with the task manager. The best way is to force the CPU to full power through the Bios.
- Memory Map above 4Gb: ON (Necessary for GPU memory)
- CPU multicore support: ON
- Intel Turbo Boost: ON
- Hyper threading:ON
- Adjacent Prefetch: OFF
- Hardware Prefetcher: OFF
- ISOC Mode: OFF
- Home Snoop Mode: Standard
- ASPM Mode: OFF (Otherwise PCIe devices are allowed to safe energy and need to much time to wake up when needed)
I recommend reading this document where the picture is taken from (page60): https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xeon-e3-1200v5-vol-1-datasheet.pdf
0 -
Hey Weckmuller.
The intel SSD 750 series is quiet old right now.
Go for Samsung PM1725a or any 8Lane SSD. Intel P4510 is also ok. Depends if you want to play uncompressed or not.
0 -
Get the latest AMD driver. They added the functionality to manually force your GPU fan to run at full speed all the time. This might help.
Do you use high quality active dapters?
Also try what Thomas said. Get a new generation GPU might also help and can be easily changed without settimg up your whole machine completly new.
0 -
Do you use high quality active adapters?
0 -
I built based on a Xeon W-2145. (Xeon w-2155 is also ok if you want to use the machines for rendering etc.)
It will run at 4,25 Ghz in all 8 core turbo mode. Note that you should populate the Ram slots with 4 memory channels. These are very important for uncompressed workflows!
If you want to spent more money: Im testing Xeon Gold 6144 and 6146 this month. They have 6 memory channels available. These are usually Dual CPU boards so they come at a certain price point. Its also a massive difference if you want to play 4K 60 or 4K0 streams. With TGA(A) the current maximum is 4 layers of 50p and 3 layers of 60p content
Keep an eye on the bios settings. I made a list of Bios tweaks here. https://watchlab.net/wiki/Tweakinglist
Win tweaking is super important but the Bios level tweaks can destroy all your work or make it succesfull. So dont forget to test that!
By the way I just tested on of these active cooled 4xm.2 PCIE adapters with and without intel VROC. Besides the WX 9100 blowing all the hot air directly inside ? Im not very happy with these devices. If you use them you should really go for enterprise ssds but also they are temperature throttling...
0 -
Did you check the following points?
https://watchlab.net/wiki/Troubleshooting_FAQ
(Happy for more advice on this!)
What EDID are you using?
0 -
Did you checked the "syncronisation chain master" button in the display settings? Without it the system will not deliver the power it has.
0 -
Some things to check:
- How long are your file names?
- How many hops (switches between production and display) does your network infrastastructure has?
- What switches are you using?
- Do you checked cables?
- I can have a quick look at the project file if u pm me.
0 -
Well I would say if you can wait then go for the new Newtek HDMI 2.0 NDI converter box and buy 4 of them.
If you need something now go for the Decklink 8K which has 4 12G SDI Inputs and put a simple HDMI to 12 G converter in front of it.
For handling 4x4K inputs I wouldprefer a WX 9100 which give you a lot more overhead but I would give it a try first...
0 -
The goal should be to have the CPU at maximum speed state and to minimize the time to ramp up the speed. I will add a little video later.
Did you set the CPU minimum frequency to 100 % in Win 10 edvanced energy settings? This should be done in every case independent from the os!
If so the CPU will still ramp up and down a little bit, but arround the maximum possible speed of 4,3 Ghz. "Overclocked" is not the right term in my opinion as people may think this is a dangerous thing. You can not overclock a Xeon CPU in the way people think about overclocking. You are setting the Intel turbo 2.0 state for the maximum speed. And that is where a CPU is at peak performance. The 4,3 Ghz are the so called " All Core Turbo" for when every core is at max level. Stay away from tools that will simulate a load on the cpu to force it for full speed. Theses tools are interfering with Watchout. CPU speed should be handled with Win or Mainboard only.
So to make it a little clearer. Usually on older CPUs the enhanced Speed Step technology will make the cpu go down to idle mode for when there is no load. For example no video playing. If you start the video the CPU needs to wake up from the halt state and speed up. This may cause stuttering playback issues. Im on Win 10 since Jan 2017 now and before on win 8.1 so my experience with win 7 is limited in this area. But I think turbo mode should always be activated. On some mainboards you can decide if the CPU or the OS is going to handle the speed shift. I prefer the OS to handle it in Win 10 as the Speed Shift Technology was developed with close cooperation with Microsoft. On Win 7 I would recommend to hand over the task to the CPU if possible. But this is speculation from my side as Im just using Win 10 LTSB for now.
These settings are super important and I saw a lot of machines with wrong settings in the past. So this is more of a general advice to please check what your bios CPU energy management is set to! And its getting worst with consumer hardware as many of theses mainboards are even more optimzed for energy savings. i think buying a LTSB license is a small price to pay considering the general price of even a cheap Watchout system. You dont have to go for the IOT version as this will also be not so easy for licensing. Win 10 Enterprise LTSB will do the job. So before throwing away Win 10 (Yes it is hard work to make it work but can be done with Pro edition) please try the LTSB version which is the perfect starting point for industrial applications.
0 -
Didnt tested with Win 7 so far but its just a Win 10 feature I guess.
What is your mainboard / CPU? You should be able to hand over the task to the cpu.
https://pureinfotech.com/windows-10-fall-update-enable-intel-speed-shift-skylake/
Never been happier than with the latest Win 10 LTSB update and AMD 2018Q2 drivers!
0 -
Of course a question of resources but we sometimes apply an additional HD output just for timecode / countdown display reasons.
Feel free to use it (50fps). Send me a pn for the simple ae project.
vimeo.com/267041786
0 -
I just want to quickly share some basic network tweaks.
When dealing with 10G and 40G networks there are settings in the network drivers which become very important for the file transfer speeds.
As the Intel X-550 series card is one of the most used cards I will explain with their settings. Most other cards use the same names for the settings:
Interrupt Moderation: OFF / The interrupt moderation prevents the CPU from getting the load from the network card. Means when to much traffic is incoming the network card will not give away load to the cpu but will simply "moderate" the incoming files to a level it can handle and also to save energy. While for many server applications this might be the right way of handling traffic especially WATCHOUT has no load on the cpu while recieving files. So you can use the resources to support the network card.
The same setting for interrupt moderation can be found in the driver under the performance options menu selection. Switch interrupt moderation: OFF
Send & recieve buffers: Set to the maximum your card can support.
RSS queues: Set to the maximum.
A good way of testing these settings is to use a simple windows network file transfer. Open up the task manager and see what happens with the cpu load with or without interrupt moderation. Multiple file transfers connected between 10 G / 40G to 10G / 40G pcs load the cpu to more then 75% on a new 8 core machine.
When handling large ammounts of data these settings can safe hours of transfer time. And while the Intel X550 is a good product also think about RDMA capable cards and manufacterers like Mellanox or ATTO when choosing the right network cards. The differences are bigger then one might think - so are the options of the network card settings which will also expand.
Didnt tested with many NDI channels so far. If anyone might share their test results in combination with input moderation settings that would be super cool.
Whatever you do keep testing!
By the way. Just finished a poduction in china where bad unshielded network cables were creating big problems. So use high quality cables for the connections! Otherwise they might create lots of problems.
0 -
As a fist formula I found out the following scenarios to work best for us:
Up to 3x layers of 4K video I use tiff because of the file size. 3x 4K tiff @1200mb/s = 3600MB/s = single 8 lanes pcie SSD or dual u.2 / pcie x4 SSD
To use 4x layers of 4K video I use tga because of the better performance 4x4K tga(a) @1600mb/s= 6400MB/s = dual pcie 8 lanes ssd
So the better performance comes with the price that you need nearly double SSD bandwith. For HD layers I count 1x4k layer as it would be 4 hd layers:
Its possible to playback 4x4K tga without problems and modern SSDs become more and more afforable.
Keep in mind that there should be at least 30% overhead to the given specs of the SSD read speed for smooth playback. Have a look at your pcie lanes!
NVME storage is also important when uploading to machines. Theres a significant difference between SATA raids and NVME while uloading!
0 -
16 hours ago, sanjeev ram said:
Hey! Welcome to the World of Watchout!
Please find my suggestions:
CPU HexaCore Intel Core i7-8700, 4500 MHz (45 x 100)
It only has 16 PCIe Lanes. Thats just enough to connect your graphics card... Get as many lanes as possible: i9 7900x has 44 Lanes. Still consumer hardware. Go for Xeons if you want reliability. Especially the Xeon W-2145 is the best combination of single core performance and number of cores. I run 8 Cores at an all core turbo of 4,25 Ghz 24/7.
Motherboard Asus ROG Strix Z370-F Gaming (4 PCI-E x1, 3 PCI-E x16, 2 M.2, 4 DDR4 DIMM, Audio, Video, Gigabit LAN)
I got nothing but problems with consumer Asus boards so far. Go for Workstation or server grade hardware! Supermicro for example or Asus Xeon Motherboards!
Options:
http://b2b.gigabyte.com/Server-Motherboard/MW51-HP0-rev-10#ov
https://www.asus.com/us/Motherboards/WS-C422-PRO-SE/
https://www.supermicro.com/products/motherboard/Xeon/C420/X11SRA.cfm
32GB should be in your machine to be future ready!
Video Card https://www.msi.com/Graphics-card/GeForce-GTX-1060-ARMOR-6G-OC.html
You will miss the EDID & sync functions. Current options AMD WX 7100, WX9100, Quaddro P5000 + S400 or NVIDIA Quaddra sync!
Capture Card https://www.blackmagicdesign.com/in/products/decklink/techspecs/W-DLK-33
And
https://www.blackmagicdesign.com/in/products/decklink/techspecs/W-DLK-31
OKI
OS Samsung SSD 850 EVO 500GB (500 GB, SATA-III)
ok!
Media WDC WD2003FZEX-00SRLA0 (2 TB, 7200 RPM, SATA-III) * 2 in RAID 0
Forget about SATA Hard Drives! NVME is the way to go and its getting cheaper every day. Stay away from m.2 and buy u.2 or PCIe. Samsung PM1725a, Intel P4500, Micron
Network Switch https://www.cisco.com/c/en/us/support/switches/sg-100d-08-8-port-gigabit-switch/model.html
OR
10 G is getting cheaper and cheaper! One option to get cheap into 10G: D-Link DXS-1100-10TS - 8 port 10GBASE-T, 2 port SFP+
QuotePower Supply https://www.corsair.com/us/en/en-eu/rmx-series-rm750x-750-watt-80-plus-gold-certified-fully-modular-psu
Cabinet https://www.antec.com/product/case/vsk4000e-u3.html
Software Windows 8.1 Professional
Well I would suggest Win 10
0 -
As written in few postings before I have a very high success with using Win 10. We are building machines for over a year now (Win 8.1 before) with it and the performance increased a lot.
To be honest. Yes it was a long and hard way to go there. But I dont wont to miss it anymore! Using a Win 7 machines feels like a time travel now.
The easier way and of course the more expensive way is to use Win 10 LTSB. Still you need to tweak it but the starting position is much better then with Win 10 Pro.
If you are based in Europe send me a PN and I can tell you where and how to get it if you dont have a dealer for it. Its not simply offered in a online store but Im happy to help!
0 -
From my experience I can unfortunately say notging but bad things about these Asus boards!
To make things better I recommend you the following Bios tweaks.
Turn off:
- Hardware Prefetcher
- Adjecent Line Prefetcher
- Intel Adaptive Thermal Monitor
- Advanced Intel Speed Step ( This is just for the older CPU Generation! With the new Xeon W and Xeon Scalable you need to turn it on)
- CPU C States
Turn on: CPU Turbo Mode
General things about Asus X299 Boards. Have a look at the PCI lanes. What is the exact configuration of your cards? How are the SSDs connected?
Do you watercool the m.2 card? This one will get into thermal throtteling very fast! I can run enterprise SSDs at 70 degrees for 24/7 but an m.2 is just rated at 40 degrees until it goes into thermal throtteling. Please stay away from m.2 disks. They are not meant to be used in such a scenario! Verified and tested from me: Intel P4500, Micron 9100 and for max speed Samsung PM 1725a (PCIe slot model for full 8x Lanes and 6200 MB/s) You can get consistend data rates for hours out of them but not from m.2 disks...
Read this:
https://www.pugetsystems.com/labs/articles/Samsung-950-Pro-M-2-Throttling-Analysis-776/
0 -
Hey People.
The current e2 firmware EDIDs seem to create problems in combination with AMD cards and WATCHOUT.
The problem can be solved by using and emulating EDIDs in the graphics cards. If someone needs EDIDs you can write me a PN.
One problem was actually solved by using an old 1080p60 edid from e2.
I always use internal EDIDs. Thats preventing lots of issues.
Jochri
0 -
I used this tool on a couple of shows:
https://www.bome.com/products/bomenet
Cross platform compatible and super easy to use.
Also in combination with their Bome Box!
0 -
Some updated informations on Win 10 versions and hardware specs:
I recently had the situation with our current machines and while building our new machines that the current Win 10 1709 does not deliver the performance I expected.
Here some links/articles:
- https://medium.com/changewindows/i-dont-want-microsoft-to-release-1709-914728ba7782
No problems with our standard win 10 image so far. But with the Fall Creators update I run into lots of performance problems.
Especially playing ProRes files. HAP doesnt seem to be influenced by this.
Google: Direct x9 + Fall Creators Upgrade.... Lots of people running into problems...
Same hardware and with just the Creators Update: No Problems with watchout.
So I would say: stay away from the fall creators update when building a Win 10 image!
Means also: If you choose XEON CPUs, which I think is the best choice now building Watchout machines, Microsoft wants you to use Win 10 Pro for Workstations.
Which means: 1709 (Fall Creators Upgrade) as this is the rollout version of Win 10 Pro for workstations. The consequence I took from this experience leaded me to the decision to go for WIN 10 LTSB. The Licensing was easier than I expected and volume licences gives you an easy path for reimaging and upgrading the machines when neccessary. I would recommend to go to a shop who is specialised in Win 10 licensing. If you want recommendations you can contact me per PN. Its nearly 3 times more expensive then Win 10 pro though.
By the way: This document shows that with Win10 Pro For Workstations you gain the right to use the enterprise version: http://www.reckzigel.at/wp-content/uploads/2017/10/Windows10ProforWorkstationsEdition.pdf
UPDATE: Just talked with the license dealer: You need a "qualifying" system to upgrade. So no free Win 10 LTSB with the Workstation Version. It will still cost you arround 300€ per LTSB License. Still compared to the troubles of activating lots of machines and building the Image I would say its worth the price...
Conclusion:
1709 Fall Creators Upgrade => Stay away from it!!!!
1607/ 1703 =>Tested and save version. We did some big shows on it.
New machines with new Intel Xeon W:
Go for the Win 10 LTSB version with Volume Licensing through Win 10 Pro for Workstations license or directly from your dealer.
Not tried yet but a possible solution to the problem if someone wants to try: https://www.drivethelife.com/windows-10/fix-directx-errors-after-windows-10-update.html
PS Some more nerdy tech stuff:
While until now Intel Speed Step technology needs to be deactivated to prevent Watchout from running out of sync on the new Xeon-W CPUs this feature must be activated to allow the CPU to go into turbo mode and use all the speed possible. Not tested for sync yet but keep you updated. Check with your specific mainboard / CPU combination.
Why changing to speed step now?:
Intel implemented a technology called speed shift: "Specific jumps in frequency are reduced to around 1ms with Speed Shift's CPU control from 20-30 ms on OS control, and going from an efficient power state to maximum performance can be done in around 35 ms, compared to around 100 ms with the legacy implementation." This is very important when syncing different machines. If this is working as promised that also gives us a significant performance boost. Minimum CPU Speed should be set to 100% in Win so in theory efficient power states should not matter to the CPU and just the jump from arround 3,9 to 4,3 Ghz can be done now in 1 ms instead of 30ms.
XEON W-2145 is the next weapon of choice. Having an all core turbo of 4,3 Ghz (measured) and trying to get even more out of it....
Some might ask why not choosing the 10-Core Xeon W? It has a slower base speed which means also a slower all core turbo.
Didnt check it myself but as the iMac Pro is using variants of the Xeon 2145 and 2155
you can see in this benchmark that the 10 core is getting significantly slower in All Core Turbo performance compared to the 8 core on full load.
The only faster CPU configuration for a mediaserver would be the XEON Gold 6144 in a dual CPU configuration which would make up for the slightly lower clock speed.
However this would cost arround 7000 $ / 6000 € per dual cpu. This would be the choice if you would use the machine also for rendering applications...
And here some theory about what I wrote
https://colfaxresearch.com/xeon-2017/
Conclusion: Current fastest Watchout CPU Xeon E5 1650 v3 / v4 and for new builds Xeon-W 2145 or Xeon Gold 6144 in dual CPU mixed usage systems.
If you dont want to go the Xeon way i think the i9 7900x with 10 cores and an all core turbo of arround 4 Ghz is the way to go. ( I would always choose the Xeons)
0 -
I agree with most of Dorians specs.
However I think m2 SSDs are getting way too hot under continous load even when there is active cooling.
The surface is just too small to handle the heat. I can recommend Intel P4500 / Micron 9100 & 9200 SSDs.
They not only give you constant throughput but also better warranty options from my experience. They are made for 24/7 use.
The intel x550 is currently the standard ethernet card of choice. You can go for atto too. There are lots of optimization options even in a network card.
Also look at this topic: http://forum.dataton.com/topic/2745-new-display-machine/
0
Crazy Output machine
in WATCHOUT FORUM
Posted
If cheap is the only thing to consider ok… You can do that. That’s what seperates toys from tools I guess…