Libretro ffmpeg

Libretro Team

By Squarepusher

Lion King running on libretro ffmpeg with around 5/6 shaders stacked - hence the low framerate (my GPU can't keep up)
Lion King running on libretro ffmpeg with around 5/6 shaders stacked – hence the low framerate (my GPU can’t keep up)

This core isn’t particularly new – maister has been dabbling on/off with a libretro ffmpeg port for a good two years now. The problem was that up until now it was never really particularly useful except for morbid curiosity.

The main achilles heel has always been that video rendering was software-based through libretro. Software-rendered video is still awfully slow compared to hardware-accelerated rendering, and launching a movie player with no hardware acceleration would definitely not compare favorably to pre-existing media players.

Now that it has made the leap to libretro GL, its usefulness has increased by a lot. The most noteworthy aspect of this core is that there is a core option enabling/disabling temporal interpolation. Through motion blur it will ‘fake’ a higher framerate in movies (fake 60fps).

The Matrix running on libretro ffmpeg with waterpaint-mudlord shader - looks like The Matrix meets Waking Life/A Scanner Darkly.
The Matrix running on libretro ffmpeg with waterpaint-mudlord shader – looks like The Matrix meets Waking Life/A Scanner Darkly. (in case you think the video quality leaves much to be desired – remember that the input source here is a low-quality SD Xvid video circa mid ’00s.

Another very appealing aspect of the ffmpeg libretro core is (of course) the mere virtue of it running inside RetroArch, which means for ports that have shader support, shader passes can be applied ontop of the image. We’re pretty confident no other movie player right now is offering 8-pass shader stacking right now – never mind it being dynamically configurable from a built-in menu. Also included (an option of most interest to otakus who like to watch anime) is ASS subtitle support.

Despite the very cool nature of this ffmpeg port, it should be noted that this is fundamentally a very backwards way of implementing a movie player. While most movie players are high-latency affairs that depend on buffering and advanced A/V synchronization strategies, this ffmpeg core instead depends on a low-latency frontend (ie. RetroArch in this case) in order to deliver good audio and video. Something which might simply be too tall an order on Android given the high-latency audio/video drivers on that platform.

Terminator 1 with bsnes-gamma-ramp applied. What you can't see is how smooth this looks with temporal interpolation turned on.
Terminator 1 with bsnes-gamma-ramp applied. What you can’t see is how smooth this looks with temporal interpolation turned on.

An attempt will be made by me to get this running on mobiles and anything in fact supporting libretro GL – this might have to involve baking in ffmpeg as a static library since on the mobile platforms ffmpeg is not available or can be installed as a dependency.

All in all, the temporal interpolation option really makes a big difference in the movies I’ve tried it with, and overall it’s an exciting and promising indicator that libretro doesn’t necessarily have to be confined to merely emulators or games.

Libretro GL – SceneWalker

Libretro Team
The SceneWalker with the Silent Hill 3 Chapel model loaded in. This entire map mostly works correctly.
The SceneWalker with the Silent Hill 3 Chapel model loaded in. This entire map mostly works correctly.

By Squarepusher

Here is the second tech demo by maister made to showcase what is possible with libretro GL. SceneWalker is a heavily modified version of ModelViewer.

Instead of loading ‘character models’, its main purpose is to load in ‘scene models’. Once loaded in, you can then walk around these environments from a first-person perspective. It’s possible to move through environments using either the D-pad and/or the analog sticks. Pressing RetroPad B button allows you to jump – this comes in handy with some models where certain obstacles are preventing you from fully traversing the environment.

A great deal of environments already work within this SceneWalker app – the initial ‘placement’ of your starting position is currently a problem in some models since it is possible for you to be ‘dropped’ inside a void space. This happens for instance with the ‘Devil May Cry 4’ street model which makes it impossible to walk around that map.

Other maps (such as Silent Hill 3’s chapel) work fine on the other hand.

Basic collision detection and gravity has been implemented – it mostly works the part for most models.

Below you’ll find some screenshots of some models that we have loaded into this SceneWalker.

This is the map 'Amnesia Fields' from the PSP version of Tekken 5: Dark Resurrection. The skybox is missing from this model but everything seems to more or less work correctly. There are some popup issues that might or might not be overcome in the future.
This is the map ‘Amnesia Fields’ from the PSP version of Tekken 5: Dark Resurrection. The skybox is missing from this model but everything seems to more or less work correctly. There are some popup issues that might or might not be overcome in the future.
This is the Twisted Corridor map from Alice In Wonderland. There are plenty of places here where if you don't carefully jump on the platforms you can fall into the void.
This is the Twisted Corridor map from Alice In Wonderland. There are plenty of places here where if you don’t carefully jump on the platforms you can fall into the void.
A pub scene from the MMO game Vindictus rendered inside the SceneWalker.
A pub scene from the MMO game Vindictus rendered inside the SceneWalker.
This is the police station main hall scene from Resident Evil: Umbrella Chronicles. The model uses DDS textures which are currently a stub - hence why there are no textures right now when showing this model.
This is the police station main hall scene from Resident Evil: Umbrella Chronicles. The model uses DDS textures which are currently a stub – hence why there are no textures right now when showing this model.

Platforms

Scene Walker right now runs on:

  • PC (Windows/UNIX/OSX)
  • Android
  • iOS
  • Blackberry QNX (BB10/Playbook)

The maximum supported internal resolution at which you can render the models depends on the platform you’re running Scene Walker on. On the mobile platforms we have consciously decided to set the maximum supported resolution at 1024×768 – this should be the native resolution of the iPad 2/iPad Mini and it is doubtful that even on a powerful tablet you’d have much need for 1080p internal resolution anyway.

On PC 1920×1600 is the maximum internal resolution at which you can render these models.

Like all the non-GL based libretro cores, you can apply any amount of shaders that you want.

Scene Model links

I’m not exactly sure how hard-ball game developers are when it comes to these models – but anyways, there is a lot of source material you can find on the Internet.

Some places that supply them is DeviantArt (search for DeviantArt + XNALara-  that should show up a bunch).

Another site that I’ve found includes a lot of useful models is this one (http://thefree3dmodels.com/). Make sure that when downloading a model form there, that it says ‘OBJ’ or something similar. Models that are not in this format can’t be expected to run right now.