Simple Howto for re-encoding (bluray) MKV's (linux)

[skittle]

This will cover my favorite method: using x264 compiled with FFmpegSource2 input directly, or if crop/resize is needed pipe from mplayer->x264. In either case, end up muxing everything together with mkvmerge gui.
I do not suggest using x264 from repository. It will likely be very outdated and not contain ffms2 input
First we need to get basic compiling dependencies (available in your repositories):
yasm, libgpac-dev, subversion, git, automake, pkg-config

Second we need to download and compile (do not use from repository) in this order: ffmpeg, ffms2, then x264:

Code: Select all

#get and compile ffmpeg and disable all but video decoders
cd /home/<usr>/source/
#svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk --revision {2010-07-01} ffmpeg #use this syntax to checkout older ffmpeg if current is broken
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --disable-everything --enable-gpl --enable-postproc --enable-protocols --enable-demuxer=matroska,ogg,avi,h264,mov,m4v,mpegts,mpegvideo --enable-decoder=h264,vc1,wmv*,mpeg2video,mpeg1video,mpeg4,theora,vp8 --enable-parser=h264,mpeg4video,mpegvideo,vc1,vp8
sudo make install

Code: Select all

#get and compile ffms2
cd /home/<usr>/source/
svn checkout http://ffmpegsource.googlecode.com/svn/trunk/ ffms2
cd ffms2
./configure
make
sudo make install

Code: Select all

#get and compile x264 with ffms2 input
cd /home/<user>/source/
git clone git://git.videolan.org/x264.git x264
cd x264
./configure
make
sudo make install

Now that x264 is compiled, you can call it directly! Here is a simple command for a standard bluray movie using preset+tuning system in recent x264:

Code: Select all

x264 <input>.mkv --preset slow --crf 18 --tune film --level 41 --colormatrix bt709 --output <output>.mkv

x264 now supports cropping and resizing natively with the switch --vf. Please read the help page for more information.

If you want more exotic filters that are not included in x264 filter code, you can use mplayer (from repo, or compiled your self) to decode and pipe cropped/resized/filtered source input to x264. This is the most common method using linux pipes:
(NOTE: if piping from mplayer/ffmpeg you do not need to compile x264 with ffms2/lavf input)

Code: Select all

mkfifo pipe.y4m
x264 pipe.y4m --preset slow --crf 18 --tune film --level 41 --colormatrix bt709 --output <output>.mkv & mplayer <input>.mkv -vf <crop/resize/filter> -benchmark -vo yuv4mpeg:file=pipe.y4m -nosound -really-quiet

When its done, use MKVmerge(gui) to put it all back together. You can find mkverge, part of mkvtoolnix here: http://www.bunkus.org/videotools/mkvtoo ... loads.html

And of course read 'x264 --fullhelp' for x264 options, and mplayer man page for more information on cropping and resizing (you can use mplayer -vf cropdetect to sort of auto cropping)

This guide was written for linux, but similar method will work in OSX/windows. (windows has to use raw stdout/stdin pipe)

update 07-03-2010: changed commandline to be more readable and avoid compile errors in ffmpeg
update 07-17-2010: x264 now supports basic crop/resize natively

[crowfax]

Legendary.

I'm just scrubbing my work laptop and putting Ubuntu 10.04 on and then I'll give this a go.

Thanks for your time & effort.

[skittle]

post if you have more questions

[crowfax]

Getting errors running the make on ffms2

Code: Select all

matthew@loki:~/source/ffms2$ make
 cd . && /bin/bash /home/matthew/source/ffms2/missing --run automake-1.11 --foreign
running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 805306365
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for working alloca.h... yes
checking for alloca... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for inflateEnd in -lz... yes
checking for CoUninitialize in -lole32... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.22... yes
checking for FFMPEG... yes
checking whether ffmpeg works... yes
checking whether -Wl,-Bsymbolic is needed... no
checking for sed... /bin/sed
configure: creating ./config.status
 /bin/bash ./config.status
config.status: creating shave/shave
config.status: creating shave/shave-libtool
config.status: creating Makefile
config.status: creating ffms2.pc
config.status: creating src/config/config.h
config.status: src/config/config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
  CXX   src/core/audiosource.o
In file included from /usr/local/include/libavutil/avutil.h:81,
                 from /usr/local/include/libavcodec/avcodec.h:30,
                 from /usr/local/include/libavformat/avformat.h:56,
                 from src/core/audiosource.h:25,
                 from src/core/audiosource.cpp:21:
/usr/local/include/libavutil/common.h: In function ‘int32_t av_clipl_int32(int64_t)’:
/usr/local/include/libavutil/common.h:154: error: ‘UINT64_C’ was not declared in this scope
make: *** [src/core/audiosource.lo] Error 1

[skittle]

This is no longer needed, ffms2 was updated to fix this issue

Hi, sorry it seems ffmpeg was updated recently and ffms2 was not. please check out slightly older revision (same date as latest ffms2) of ffmpeg:

Code: Select all

#no longer needed, ffms2 was updated to support latest ffmpeg, but if it breaks again use this syntax to get revision that works
#svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk --revision {2010-04-21} ffmpeg
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg

I confirmed this on my machine as well, older revision of ffmpeg works. ffms2 dev has been notified.[/size]

[crowfax]

Thanks. I'll give it a spin after my coffee!

[crowfax]

Everything compiled (I had to get automake from my repo's last night, forgot to mention in my previous post) and it seems to be working, it's processing the MKV now.

Is there a similar method for DVD's?

[skittle]

of course! ffms2 can accept almost any input (anything ffmpeg can). might want to pipe from mplayer, or avisynth+wine+avs2yuv to do filtering as DVDs sometimes have weird telecine and such.
Also I forgot to mention that ffms2 barfs on .m2ts and .ts so use pipes if your source is ts or m2ts.

[crowfax]

Is the mkfifo pipe.y4m command needed if all I'm running is

Code: Select all

x264 <input>.mkv --preset slow --crf 18 --tune film --colormatrix bt709 --output <output>.mkv

Lastly, just so I'm clear in my head. Once the process is done, I put all the files into MKVmergeGUI and just untick the uncompressed stream and set the compressed stream as default and let it spit out a new file, right?

[skittle]

mkfifo is only for piping from mplayer. The command you posted will work just fine, and it should use ffms2 as default.
if you want to do a quick test, add --frames 1000 --preset ultrafast also note --tune should only be used if your source matches the tune option. (ie --tune film for film, animation for cell animation etc, grain for grainy film)
read x264 --fullhelp for more information

Edit: yes, once all is done pull all into mkvmergegui and untick the unwanted streams (muxing will take a long time)

Edit2: sorry didnt realize i had mkfifo in the first command in my post :p

[skittle]

just an update, no longer needed to get older revision of ffmpeg, ffmpegsource was updated to fix the issue http://code.google.com/p/ffmpegsource/source/list

[lingenfr]

I followed a suggestion to this thread. I got all of the tools set up as per the instructions. I started with disc 2 of Fringe Season 1, a blu-ray/BD-Live disc which I ripped with makemkv. I ran the following command:

Code: Select all

x264 title00.mkv --preset slow --crf 18 --tune film --colormatrix bt709 --output fringe_s01e05.mkv

It took my 6.9gb file from makemkv and cut it down to a 1.6gb. The video appeared good, but there was no audio. I tried mkvmerge, but had no audio to merge. It ran but didn't appear to do anything. I imagine there is still quite a lot of shorthand in the original instructions that a noob is missing. As I wanted a lower resolution/smaller file anyway, I tried the next instructions to pipe to mplayer and got the following results:

Code: Select all

lingenfr@MPC-MBR:~/Videos/Fringe_Season_1_Disc_2$ mkfifo pipe.y4m
lingenfr@MPC-MBR:~/Videos/Fringe_Season_1_Disc_2$ x264 pipe.y4m --preset slow --crf 18 --tune film --colormatrix bt709 --output fringe_s01e05.mkv & mplayer title00.mkv -vf crop=1920:1072:0:4 scale=320:180 -benchmark -vo yuv4mpeg:file=pipe.y4m -nosound -really-quiet
[5] 11821
mplayer: could not connect to socket
mplayer: No such file or directory
open: No such file or directory
open: No such file or directory
DMO dll supports VO Optimizations 0 1
DMO dll might use previous sample when requested
Decoder supports the following formats: YV12 YUY2 UYVY YVYU RGB8 RGB555 RGB565 RGB24 RGB32 
Decoder is capable of YUV output (flags 0x1b)
Using SSE optimized IMDCT transform
Using MMX optimized resampler

It created a file with size of 0 that did nothing. I would appreciate some assistance to tell me what I am missing. Thanks.

[skittle]

Hi! Good to hear x264 works in the first case! regarding mkvmerge: Drag BOTH x264 output and the original MakeMKV source file into the MKVmerge(gui). Then just select to keep x264output, and audio from original then mux to a new mkv

regarding mplayer, your command line is wrong!

$mplayer title00.mkv -vf crop=1920:1072:0:4 scale=320:180 -benchmark -vo yuv4mpeg:file=pipe.y4m -nosound -really-quiet

should be:

$mplayer title00.mkv -vf crop=1920:1072:0:4,scale=320:180 -benchmark -vo yuv4mpeg:file=pipe.y4m -nosound -really-quiet

Code: Select all

$mplayer -vf help
$-vf <filter1[=parameter1:parameter2:...],filter2,...>

(also i forgot to mention/putinto commandline in my original post --level)
I have revised the command line to contain --level 41
(level 4.1 is recommended for PC and hardware playback)

If you have multithreaded mplayer adding -lavdopts threads=4 really helps too!
Hope this helps and please please read man/help pages!

[anothergene]

just a quick question. why not use something like HandBreak?

[crowfax]

The last supported version of Handbrake came out Nov 23, 2009. That makes it a dinosaur in software terms.