BestSource 2 – Frame Accurate Seeking in Everything!

Many of you VapourSynth and Avisynth users are probably familiar with FFMS2, the reasonably useful source plugin that can open almost everything and fairly often seeks properly. Or at least it used to seek fairly well back when it was released 17 years ago. Since then the number of seeking workarounds and complexity has reached insane levels due to many newer video codecs having more and more complex reordering with invisible frames and other tricks. This broke every single original assumption made about seeking. And then the new assumptions were broken. And the new new assumptions failed. And the shiniest and freshest assumptions produced the wrong frames before finally ending at the current state of not being completely broken again.

Due to the insane tedium of trying to find new workarounds I lost interest in trying to fix FFMS2. Instead I started on BestSource. The first official release 2 years ago did pure linear decoding only with some caching and tricks behind the scenes to allow for relatively fast jumping around (by linear decoding standards). It was correct but for high resolution files over 5 minutes it was not enjoyable to use. It definitely had its niche use cases but FFMS2 still had hardcore followers for a reason.

That was however just where the project stopped due to a lack of time. The plan was always to implement seeking by hashing the output frames and using that to figure out where seeking actually takes us. And now it’s happened. Perfect and fast seeking in almost all formats and containers. Complete with proper RFF handling and in most cases the index only takes up 9 bytes per frame where 8 of them is the hash itself. FFMS2 is now officially obsolete.

But why didn’t anyone do it earlier? The idea is obvious. Many FFMS2 contributors with sufficient coding skills have thought about doing it. And after you’ve accepted an indexing pass it’s a no-brainer! Except that back in 2007 it would’ve been unacceptably slow. Even parsing a whole movie to index it generated complaints because hard drives were true snails. Doing a full decode of every frame would’ve ensured nobody wanted to use it. Besides, handling the delay introduced by b-frames wasn’t hard and only mpeg4 asp and h.264 existed that had to be tested for this complexity. For a short time it was possible to both be fast and correct. Apart from the poor and unreliable seeking in FFmpeg’s mpeg/mpegts demuxers which would surely be fixed soon and then all popular containers would work well! (It has to this day not been improved at all)

R64 – New things break old things

This release fixes a lot of new things, Zen4 CPUs, Cython 3.x and OSX (not so new) compilation broken in R63.

It’s also the end for 32 bit binaries compiled by me. These binaries get almost no downloads and x64 was first introduced 24 years ago. At this point all x86 fans have a 64 bit CPU. And if you don’t you probably shouldn’t be processing video at all. 32 bit builds are also a lot less useful now that most Avisynth plugins have either been ported or have 64 bit support.

Next thing to go will probably be optimizations for x86 CPUs with less than AVX2. It’s already been 10 years since AVX2 was introduced and 8 years since all CPUs have shipped with support for it. These are more or less the same numbers SSE2 had back when I originally released VapourSynth with a SSE2 minimum requirement.

I bet many doom9 users will rage when that day comes, just like when XP support was removed.

VSRepo – A simple package repository for VapourSynth

I’ve spent the past two weeks writing a simple package manager for VapourSynth called VSRepo. It can already install most common plugins and scripts with all of their dependencies. Installing havsfunc is no longer a treasure hunt across the internet!

Go visit the doom9 thread where the link to the latest test versions are posted if you want to try it out or help add more plugins and scripts. The plan is to include it in the installer with the next VapourSynth release.

IMWRI – Making it suck less

I’ve mentioned that I hate ImageMagick before. I still hate it but finally took the time to update AND test it for ImageMagick 7. As a result it’s now a mostly usable image source or at least should be. There are only a few remaining quirks which will probably exist for all time. Like being able to figure out if a source file is floating point for any remotely relevant format apart from TIFF. But enough about that, all 8-31 bit images will be assumed to be integer and all 32 bit images will be assumed to be float. Unless it’s in TIFF in which case it will all just magickally work.

Binaries and discussion are in this thread.

Seal holding its breath to avoid the smell of ImageMagick

Blindly Copying Avisynth Considered Harmful

Some Avisynth filters are really, really popular and show up many scripts. Some of them like RemoveGrain and TemporalSoften have even been ported because they’re useful and used in so many scripts.

…But what is TemporalSoften really used for?

TemporalSoften is the definition of cargo cult programming, its code and concept gets copied over and over again and nobody knows exactly why. Let’s start with a few facts:

  • TemporalSoften is a horribly bad temporal smoother even compared to its competition over 10 years ago
  • TemporalSoften has never been used to unironically denoise anything the past 10 years

With these facts in mind you have to wonder why anyone’d ever put it in a script from the past decade. The answer is simple, if you set the denoising thresholds to their max values you get a filter that averages an odd number of frames. If you’re creative you can blend several frames with different weights. That’s actually kinda useful sometimes. So useful someone already wrote an Avisynth plugin for it. It’s a shame that it was fairly quickly forgotten. Unfortunately nobody who ported TemporalSoften to VapourSynth stopped to think about what it was actually used for and instead a very optimized version of TemporalSoften was created. And rewritten. And optimized a bit more. Because why not?

To try to counter this madness I’ve made a collection of small useful and experimental functions available. The first version can be found in this thread. Go try it and stop blending frames with TemporalSoften.

Looking for contract work

I am currently looking for contract work so now’s the chance for you to hire a good developer. My main skills are C, C++ and x86 assembler. I also have experience writing multithreaded and cross platform applications.

So why should you hire me and not someone else with a similar list?

  • I want to build up a positive reputation and the best way to do so is to have satisfied clients – if I don’t think I can do a job well I will simply turn it down instead of doing it badly
  • Used to working on tight schedules in the corporate world
  • I write clear and maintanable code – check out the vapoursynth source code for an example

Don’t hesitate to contact me at fredrik.mellbin [at] gmail.com if you’re interested.

Positive Thinking – How to Convince Yourself That You Can Code

Previously I’ve discussed how computers work so now it’s time to take a look at how humans work. I personally think this is something programming textbooks neglect to take into consideration, for example humans generally work very badly when confronted with binary operations and bit-wise shifts the first time. It can also be somewhat detrimental to their mental state. So this post is about negative thinking and what You Can Do To Break The Pattern! My personal trick is to always keep the law of attraction in mind when I code and so should you! Get the positive thinking going by putting some comments telling yourself that YOU CAN DO IT! or maybe that it’s simple. See this example from Avisynth:

if (!(audio && video)) { // Hey - simple!!
if (audio) {
  return new DirectShowSource(filename, _avg_time_per_frame, seekmode, true , false,
    args[5].AsBool(false), _media, _timeout, _frames, log, env);

And indeed that is simple! Just imagine what this technique can do for your mental health. No more feeling like a rejected PHP coder, instead you’ll be the coolest PHP coder in a basement within two blocks. Guaranteed or your money back!

Personally I prefer to apply the law of attraction in a slightly different way. For example I write my goal in every single comment. See this code snippet:

// Buy milk
for (int x = 0; x < width; x++) {
// Clean thoughts
// (representing my desire to have a clean toilet)
  dstp[x] = srcp[x-1] + srcp[x+1] + srcp[x-1-stride] + srcp[x+1-stride] + srcp[x-1+stride] + srcp[x+1+stride];
  // For loops are trivial and even my grandma could do them...
  // if she was still alive

Sometimes I try this but I have had less success with it:

// Hire Fredrik Mellbin, he's awesome

I’m currently experimenting with writing it every 10 lines in my code just to see what happens. I’ll post a follow-up later and tell you all how it works out.

Open Binary – Introducing a Practical Alternative to Open Source

I’ve been thinking about not only announcing releases and features, but also discuss (read: point and laugh at) some very common annoyances. I hope to one day be seen as a meaner alternative to The Daily WTF but with less free mugs and more open source. Anyway, on to this post’s subject…

Open Binary – source code so obfuscated, “optimized” and arcane that despite an open source license nobody can edit or benefit from reading it. Your only hope is to compile it into a binary and hope it works. There are plenty of examples of this in the video world, such as mplayer, most popular Avisynth filters and to be honest almost every single piece of code written in the field of video processing.

So how do you produce an Open Binary? Well, in my opinion you have to put effort into multiple levels to succeed. For example one important step is to OPTIMIZE! And by that I mean bitwise shifts! No compiler can ever figure out that a/2 can be compiled to a right shift so you have to help it, it also makes the code faster. Another important detail to know about CPUs, even the most modern ones, are that they are slow readers. Armed with this knowledge make all variable names short so there’s less to read for the poor CPU. For text parsing we can actually do one better since modern CPUs are good at numbers, simply use the ascii code instead of the letter in any text operation.

An example of proper text parsing taken from TIVTC:

if (*linep != 0)
{
	qt = -1;
	d2vmarked = false;
	*linep++;
	q = *linep;
	if (q == 112) q = 0;
	else if (q == 99) q = 1;
	else if (q == 110) q = 2;
	else if (q == 98) q = 3;
	else if (q == 117) q = 4;
	else if (q == 108) q = 5;
	else if (q == 104) q = 6;
	else
	{
		fclose(f);
		f = NULL;
		env->ThrowError("TFM:  input file error (invalid match specifier)!");
	}
	*linep++;
	*linep++;
...continued for several hundred lines

There are several other techniques you can use too, for example writing pure assembler, or even better, inline assembler which effectively will tie all your code to one platform and compiler too at the same time in addition to being near impossible for anyone to modify or understand! You can also play the shell game with pointers and global variables, have one function add an offset to a pointer and pass it to the next which subtracts it again. The secret it is to put spaghetti in your sauce, so to say.

So who should use this approach to leverage the open source benefits? Big evil companies of course! Sure, you’ll have to reveal the source code but no one can ever use it for anything anyway. This is the end of part one of my “Business Strategies for the Modern Monopolist” series. I’ll be posting part two shortly.

Here’s a final example of successful use of assembler only to make a true Open Binary, again from TIVTC as I’ve spent far too much time staring at it recently. The actual post ends here so you don’t have to scroll down to look for more.

__asm
{
	mov y, 2
yloop:
	mov ecx, y0a
	mov edx, y1a
	cmp ecx, edx
	je xloop_pre
	mov eax, y
	cmp eax, ecx
	jl xloop_pre
	cmp eax, edx
	jle end_yloop
xloop_pre:
	mov esi, incl
	mov ebx, startx
	mov edi, mapp
	mov edx, mapn
	mov ecx, stopx
xloop:
	movzx eax, BYTE PTR [edi+ebx]
	shl eax, 3
	add al, BYTE PTR [edx+ebx]
	jnz b1
	add ebx, esi
	cmp ebx, ecx
	jl xloop
	jmp end_yloop
b1:
	mov edx, curf
	mov edi, curpf
	movzx ecx, BYTE PTR[edx+ebx]
	movzx esi, BYTE PTR[edi+ebx]
	shl ecx, 2
	mov edx, curnf
	add ecx, esi
	mov edi, prvpf
	movzx esi, BYTE PTR[edx+ebx]
	movzx edx, BYTE PTR[edi+ebx]
	add ecx, esi
	mov edi, prvnf
	movzx esi, BYTE PTR[edi+ebx]
	add edx, esi
	mov edi, edx
	add edx, edx
	sub edi, ecx
	add edx, edi
	jge b3
	neg edx
b3:
	cmp edx, 23
	jle p3
	test eax, 9
	jz p1
	add accumPc, edx
p1:
	cmp edx, 42
	jle p3
	test eax, 18
	jz p2
	add accumPm, edx
p2:
	test eax, 36
	jz p3
	add accumPml, edx
p3:
	mov edi, nxtpf
	mov esi, nxtnf
	movzx edx, BYTE PTR[edi+ebx]
	movzx edi, BYTE PTR[esi+ebx]
	add edx, edi
	mov esi, edx
	add edx, edx
	sub esi, ecx
	add edx, esi
	jge b2
	neg edx
b2:
	cmp edx, 23
	jle p6
	test eax, 9
	jz p4
	add accumNc, edx
p4:
	cmp edx, 42
	jle p6
	test eax, 18
	jz p5
	add accumNm, edx
p5:
	test eax, 36
	jz p6
	add accumNml, edx
p6:
	mov esi, incl
	mov ecx, stopx
	mov edi, mapp
	add ebx, esi
	mov edx, mapn
	cmp ebx, ecx
	jl xloop
end_yloop:
	mov esi, Height
	mov eax, prvf_pitch
	mov ebx, curf_pitch
	mov ecx, nxtf_pitch
	mov edi, map_pitch
	sub esi, 2
	add y, 2
	add mapp, edi
	add prvpf, eax
	add curpf, ebx
	add prvnf, eax
	add curf, ebx
	add nxtpf, ecx
	add curnf, ebx
	add nxtnf, ecx
	add mapn, edi
	cmp y, esi
	jl yloop
}