HAP Encoding not looking so good in FFMPEG

[keitht]

I'm trying to understand why I am getting substantially worse results while encoding to HAP using FFMPEG vs. Adobe Media Encoder.  My problem is I need to do Chuncks encoding (which cannot be set using AME) but the results from FFMPEG are not up to quality.

See results here:

https://www.dropbox.com/s/4fakivsytefl60c/HAP%20Test.tif?dl=0 

 

Has anyone else seen these issues?

 

Here is my FF command line:

ffmpeg.exe -i "H:\_Render\test.mov" -vcodec hap -format hap_q "H:\_Render\output.mov"

 

AME is set to HAP Q, Quality=100, Render at Max Depth (not that any of those settings makes a difference if my understanding is correct).

 

Any ideas on how I can get the matching quality from FFMPEG or how ELSE I can do HAP encoding with Chunks ON WINDOWS?

 

-kt

[Benoit]

Hi Keitht,

 

Did you try to encode with ffmpeg from still images? We often use ffmpeg to encode HAP (not Q) without any quality issues...

 

Benoit

[keitht]

No, I have not tried coming from image sequences (that's what you mean, right?) - but that should not make a difference on encode quality I would think.  In any case, my situation makes it not possible for me to input sequences anyways.

 

FYI - What I have learned from the HAP forum is that AME (and AE etc...) use the quicktime codec written by hap to encode.  FFMPEG does NOT... and wrote their own encoder.  I, for one, am not happy with the FFMPEG encoding.  Anyone else?

 

-kt

[Benoit]

Your quality issue could come from other things. My guess is ffmpeg decode prores 422 with quality loss and your issue comes more from the decoding than the encoding…

 

Benoit.

[keitht]

Your quality issue could come from other things. My guess is ffmpeg decode prores 422 with quality loss and your issue comes more from the decoding than the encoding…

 

Benoit.

 

That's an interesting thought.  I'll test to be sure... but since the same input file gets excellent results with FFMPEG for 4K MPG2 I would say that decoding is not the issue.  I'll run the test anyways.

-kt

[Miro]

Hi, there are three prores codecs in FFmpeg (prores, prores_aw and prores_ks) and the default one is no good. Try the prores_ks instead. Also use the -chunks option to specify the number of chunks for multi-threaded decoding.

 

//Miro

[keitht]

Hi, there are three prores codecs in FFmpeg (prores, prores_aw and prores_ks) and the default one is no good. Try the prores_ks instead. Also use the -chunks option to specify the number of chunks for multi-threaded decoding.

 

//Miro

I dont understand?  We are talking about HAP encoding here.  Not ProRes.  The ProRes in this case is the source and look good.

 

-kt

[JJ Myers]

I myself am planning on using ffmpeg for future HAP encoding, considering that anything using the 32b Quicktime component on OS X  will be reaching EOL very soon. In fact, the latest Adobe AME and AE OS X releases have already pulled HAP from the list in anticipation of the next version of OS X (which will no longer support any 32b components). There has been a backlash directed towards Adobe from the live events community concerning the lack of warning to users and the lack of planning to have a solution in place ahead of time. Adobe has mentioned on various forums that they hear the response and intend to provide a solution in the future.... for whatever that's worth.

I am sorry Keith I do not have any results to share with you at this time, as I am just getting started. I will indeed let you know what I find out as I experiment. I am hopeful in finding a way to get acceptable results out of ffmpeg, because the project really seems to have taken off and is a great contribution to the development community!

[JJ Myers]

Hey Keith - we have recently been doing some testing with ffmpeg and have found that our output files display significant frame loss when decoded in conventional players (Quicktime) or even no decoding whatsoever (VLC), but however look great in WO... or at least on par with files that we otherwise produce out of a traditional After Effects render queue. Is that consistent with your findings? Or are your remarks in this thread regarding quality more along the lines of things like color reproduction? Just curious. If the quality concerns with ffmpeg is really just a challenge in terms of accurate preview outside of WO, it certainly seems like more of a solvable issue. As in, someone just needs to build a decent preview application for HAP files that we can provide clients and vendors.

[keitht]

Hey JJ.  Sorry - for some reason I was didn’t get a notification on the reply to this thread - I am just seeing it now.

It has been a year or so since I have done real world testing on this.  I am not sure if anything has changed.  My finding were that encoding to HAP with FFMPEG resulted in worse quality than if encoded with AME.  The differences are in banding, color depth and artifacts (blocky compression).  It was by no means unacceptable quality, but definitely worse for computer generated content. 

I am not sure I understand what you are saying.  Playback performance is fine.  Of course playback performance in QT player or AE is not real-time.  Quality in WO seemed to match quality in AE.  Meaning I notice a difference when playing the 2 files side by side in WO, and when I compared Frame-by-frame, zooming in in AE the exact differences are easier to pinpoint, but I think are the same I see in WO.

For this reason, I have stayed on AME 12.0.1 and have been making HAPs there will good results.
FYI - AfterCodecs just uses FFMPEG and others have noticed the difference too: 

Here is my original report and confirmation that different encoding software WILL yield different results:
https://github.com/Vidvox/hap-qt-codec/issues/39

I am sure we are all tracing what is happening with HAP and Adobe products very closely these days.  Here are a few relevant links I have been tracking.  Many I have not yet tested - so maybe all could chime in there findings and other possible solutions until this mess settles down:

The D3 guys have written a plugin for Premiere (that should work in AME).  I haven’t personally tested yet.
https://github.com/disguise-one/hap-encoder-adobe-cc/releases

Apparently Adobe is at least listening.  I will try to test next week.  The comments below do not sound as though it is working, but this thread claims that the newest AE will at least be able to use HAP footage.
https://adobe-video.uservoice.com/forums/911311-after-effects/suggestions/33853372-support-the-hap-codec

What about using HAP AVI (rather than QT).  Will this work in WO?  I am guessing not, but perhaps this is the area Dataton should be moving to??
https://support.troikatronix.com/support/solutions/articles/13000044970-create-windows-native-hap-avi-video-files
http://renderheads.com/product/hap-for-directshow/
 

Does anyone have any newer/better information to add here?

Keith Tromer
Corporate Imaging

[WatchDog]

Hello Keith,

I have used the following software to export Hap out of AE: https://www.mekajiki.com/rendergarden/ It is similar to BG render since you can have multiple comps rendering at once, or even segments of the same comp rendering at once. It touts "network" rendering as well over multiple computers, hence the "Garden" in the name. It is very similar to a render "farm". It relies on FFMpeg for Hap encoding and that is done automatically after the initial render is done. I put the software through the wringer and it rendered faster, by at least 25%, than AE's native renderer. That includes the post render encoding to Hap. So I have rendered and encoded with this AE plugin faster than I could with AE alone doing Hap natively. The reason for the processing efficiency is that it utilizes multiple cores for rendering. I see this product as the only current viable alternative to Adobe's drop of Hap support.

Going back to your concern of After Codec's  FFMpeg poor quality Hap encodes, I have experienced the same quality loss using After Codecs. I did not have the same experience using Render Garden. Perhaps its native settings produce a better Hap encode? I know I had the ability to customize the encode to an extent using FFMpegs command line within the GUI, so perhaps that was it. I did not notice, to the degree you speak of, a significant quality difference between HAP encoding natively out of AE vs Render Garden's use of FFMpeg, except for a slight gamma shift. I did notice that any attempt to play back the resulting FFMpeg based Hap file through QT/VLC on a mac/pc resulted in consistent frame loss during playback. My 30fps file could only playback at 15fps. In contrast, those same Hap files played back fine in WO. I did side-by-side comparisons of AE's native Hap encoding vs Render Garden's to scrutinize quality/playback and did not notice a difference. Not sure what is going on under the hood with the FFMpeg based Hap files producing choppy playback with QT and other media players, but it is a non-issue as they play back fine in WatchOut since the servers do not rely on QT for decoding.

My attempt to use Disguise's Alpha Hap encoder was not very successful in terms of actual use with AE, so not a viable alternative just yet. Adobe used Disguise's progress as a means to consider the "support Hap" case closed. The only thing Adobe did was add Hap decoding, so you can import Hap into AE. As far as encoding Hap, they left that to the 3rd party effort from Disguise. I do appreciate Disguise's timely effort, but it is still "Alpha" at best.

Render Garden seems to be the best alternative to date. Like you, however, I'm sticking with a version of AE that still supports encoding Hap until I am forced to seek out paying for 3rd party solutions. I do not expect any further development or support from Adobe, so the clock is ticking.

[keitht]

Good info WatchDog.  I had not tried Render Garden, because it uses FF.  Gonna have to look into it and see if they are using different switches than I am or a different build that uses different encoding.

I was able to try the Disguise plugin.  I have bee on site, so remoting in to office machine so as not to mess anything up on site.  Therefore although I was able to test the process, I am unable to adequately rate the quality/performance of the result yet.  However - I WAS able to get it installed in CC 2019.  And I DID have AE (que to AME) make a HAP file - then reimported into AE.  Seems to work!!   Also, I have contributed a few messages to their development Git hub and:  1. They are very active, this is going to get better, fast I think.  2.  They also see the difference between quality of FFMEG... but also the same big speed improvement you mentioned.

I would love to see if you can show a difference in quality running the same footage through FF direct and RenderGarden.

Lastly in regards to your playback issues... they are not an issue.  Although QT and VLC can read the HAP files, they do not use the GPU for decoding, therefore cannot playback in real-time (depending on the size of the media of course).

What trouble did you have getting the Disguise plugin to work?  Perhaps what I did was different.

-Keith

 

[WatchDog]

It has been a few months now since I went through a significant amount of render tests, which included side-by-side playback in WatchOut. I know I used FFMpeg natively and in conjunction with Render Garden for tests, including AE's encoding of Hap. I may have thrown in AVF's batch encoding as well. I'd have to revive my findings for definitive results, but I concluded that tweaking some parameters in the command line entries within Render Garden's GUI resulted in acceptable Hap files. I had to abandon finalizing my results so I could focus on existing projects.

I do note your mention of QT/VLC playback not using GPU for decoding. I only point out the result because AE's native Hap decoder ala QT plays back with very few dropped fames in QT/VLC. That same video encoded via FFMpeg does not ala Render Garden does not. It is so bad that it looks like stop motion photography. I bring it up because there is something going on under the hood that is very different than what we have been accustomed to with AE Hap files, as far as a quick QC through QT or VLC. Initially I thought FFMpeg was producing junk encodes, but was quickly proved wrong when I was able to play back the file flawlessly in WO.

For me, the problem with using the Disguise solution is that it farms out renders to AME, which makes the multi-render workflow far less efficient and effectively terminates the use  of output modules. In short, it adds unnecessary steps into a workflow that knocks efficiency back 5 years. AME also crashed multiple times during initial tests, although I have not tested with the latest Adobe CC version, nor do I plan to migrate to any new version until this debacle is resolved to my satisfaction.

Feel free to test out Render Garden. They do have a 7 day trial. Other than what was noted above, I did see a slight reduction in file size as well. The only reason I am not currently using Render Garden is that I'd have to purchase a license for every work station while simply reverting back to AE "pre-hap support drop" works seamlessly for free. I'm just waiting to see what happens at this point since this is a new issue for our industry and great minds are on it.

[keitht]

All very interesting.  I usually tend to batch/multi-processor render from AE to uncompressed first anyways, so - for me - the AME plugin is a fine (even preferred) solution.

I hope to test the Disguise Plugin further this coming week.