Shader Changes

Abstract

GLSL shaders now preferred over Cg when possible
Update to latest RetroArch for compatibility with updated GLSL shaders

Cg shaders demoted, GLSL promoted to first-class

Portability and compatibility are major goals for RetroArch and libretro, so we invested heavily in Nvidia’s Cg shader language, which worked natively anywhere their Cg Toolkit framework was available (that is, Windows, Linux and Mac OS X), as well as on PS3 and Vita, and could be machine-compiled to messy-but-usable GLSL (lacking a few features, such as runtime parameters) for platforms that lacked the framework (primarily ARM / mobile platforms). Cg was also so close to Microsoft’s HLSL shader language that many Cg shaders will compile successfully with HLSL compilers, such as those available with Windows’ D3D driver and on Xbox 360.

This was great for us because we could write shaders once and have them work pretty much everywhere.
Sadly, Nvidia deprecated the Cg language in 2012, which left us in a bad spot. Since then, we’ve been limping along with the same strategy as before, but with the uneasy understanding that Nvidia could stop supplying their Cg Toolkit framework at any time. Rather than sit idly by, waiting for that other shoe to drop, we took it upon ourselves to hand-convert the vast majority of our Cg shaders to native GLSL with all of the bells and whistles. TroggleMonkey’s monstrous masterpiece, CRT-Royale, still has a couple of bugs but is mostly working, along with its popular BVM-styled variant from user Kurozumi. Additionally, before this conversion, many of our Cg shaders were flaky or completely unusable on libretro-gl cores, such as Beetle-PSX-HW’s OpenGL renderer, but these native GLSL conversions should work reliably and consistently with any core/context except for those that require Vulkan (namely, ParaLLEl-N64’s and Beetle-PSX-HW’s Vulkan renderers).

With the GLSL shaders brought up to speed, we can finally join Nvidia in deprecating Cg, though it will still remain as an option–that is, we’re not *removing* support for Cg shaders or contexts at this point–and we will continue to use it where there is no other choice; namely, Windows’ D3D driver and the Xbox 360, PS3 and Vita ports. Moving forward, our focus for shaders will be on native GLSL and our slang/Vulkan formats, though we will likely still port some to Cg from time to time.

RetroArch now correctly handles #version directives in GLSL shaders; GLSL shader repo updated to match

There have been a number of updates to the GLSL shader language/spec over its long life, and shader authors can use #version directives (that is, a line at the top of the shader that says #version 130 or whatever) to tell compilers which flavor/version of GLSL is required for that shader. However, RetroArch has long had a strange behavior whereby it injected a couple of lines at the beginning of all GLSL shader files at compile time, and this broke any shader that attempted to use a #version directive, since those directives must be on the first line of the shader. This meant that our shaders couldn’t use #version directives at all, and all of our shaders lacked #version directives until very recently for this reason. These #version-less GLSL shaders are still perfectly compliant GLSL because GLSL v1.10 didn’t support directives, either, but the necessity of leaving off the #version started to cause some problems as we whipped our GLSL shader library into shape.

The error caused by adding a #version directive under the old behavior.

On AMD and Nvidia GPUs, the compilers would just toss up a warning about the missing directive and still expose whatever GLSL features were available to the GPU, which worked out great. On Intel IGPs, however, the compiler tosses the error and then reverts to only exposing the features available in ancient GLSL v1.10 (released way back in 2004). As a stopgap, we gave many shaders fallback codepaths that would still work in these circumstances, but a number of other shaders were either impossible to make compatible or even the compatible result was imperfect.

So, as of this commit (courtesy of aliaspider), RetroArch will no longer reject shaders with explicit #version directives, and we have added those directives to any shaders that require them at the lowest version that still compiles/functions properly. That is, if the shader doesn’t use any features that require greater than #version 110, they will still have no #version specified, and any shader that requires #version 120 but not #version 130 will not have its requirements increased to the higher version for no reason. This should keep our GLSL shaders as compatible as possible with older hardware, and including the #versions explicitly when needed will also make it easier for other programs/developers to utilize our shaders without any unnecessary guesswork due to behind-the-scenes magic.

This change does require a clean break, insofar as older versions of RetroArch will choke on the new #version directives (that is, they’ll fail to compile with the “#version must occur before any other program statement” error pictured above), so users with Nvidia or AMD GPUs must update their RetroArch installation if they want to use the updated shaders. Users with Intel IGPs will be no worse off if they don’t update, since those shaders were already broken for them, but they’ll probably *want* to update to gain access to the many fancy shaders that now work properly on their machines.

Mobile GPUs using GLES had many of the same issues that Intel IGPs had, with many shaders refusing to work without #version directives, but GLES compatibility added in a further complication: GLES requires its own separate #version directives, either #version 100 es or #version 300 es, which are different from and incompatible with desktop GL’s #versions. To get around this, we added a trick in RetroArch to change any #version of 120 or below to #version 100, which is roughly comparable in features to 120, and any #version 130 or above to #version 300 es whenever a GLES context is used. This should get everything working as effectively and consistently as possible on mobile GPUs, but if anything slipped through the cracks, be sure to file an issue report at the GLSL shader repo.

RetroArch v0.9.9 Coming Soon

It’s almost time for a new release of RetroArch, and there a number of big changes coming up. First of all, RetroArch 0.9.9 will mark the release of RetroArch on iOS and Blackberry 10/Playbook Tablet OS. These ports were made possible with the help of CatalystG and meancoot, respectively – for which many thanks. The iOS port will be released on Cydia and on our forum. It is possible to run it on a non-jailbroken device – but it will require that you are able to code-sign yourself (ie. if you are a registered Apple developer with the ability to code sign).

PCSX ReARMed on iOS

For iOS, perhaps the single biggest hurdle was getting PCSX-ReARMed working, which required notaz to rewrite much of the assembly code to work with Apple’s ancient GAS assembler version (big thanks to him for that!). With that completed, this should be the first time PCSX ReARMed will appear on iOS – through RetroArch.

RGUI

Elsewhere, Themaister and Squarepusher have been toiling away at a million other features, including the promotion of RGUI to a robust and feature-filled in-game menu system for the platforms that otherwise lacked such a thing, particularly the PC platform (i.e., Windows, Mac OS X and Linux). From its humble beginnings with the Gamecube/Wii port, RGUI now provides a way to change emulation cores, swap out ROMs, configure shaders and more, all without leaving the fullscreen gaming interface:

rgui

Cave Story (NXEngine)

ToadKing and Squarepusher have also done some work on ‘uncrippling’ Cave Story (ie. NXEngine). Previously, the file I/O would make it unbearably slow on consoles. This has mostly been fixed now that everything is pre-cached into RAM at initial startup. There are still some incidental dips to 59.50fps and 59.2fps, though, which causes some sound stuttering. The cause of these dips is still unknown but we feel that–compared to before–NXEngine can be safely released on consoles now without being an utter embarrassment. “Xbox 1/360 will require some further patching up of the codebase because NX Engine did some global symbol table trickery and the MSVC linkers have the (oh so ‘smart’) tendency to ‘strip away’ unreferenced symbols as an ‘optimization feature’ with no way to stop it from doing that (even /ref:noopt doesn’t help there),” Squarepusher noted.

TyrQuake

The port of TyrQuake will also be bundled with RetroArch 0.9.9. A lot of work went into making it work on Xbox 1 and Xbox 360 – including making the C99 codebase cross-compilable as C++98 and (for Xbox 1) resorting to a hacked-up template ‘typeof’ implementation (ye, don’t ask) for MSVC 2003. “I also threw in some additional ‘hackish’ features like ‘dither filtering’ (borrowed it from some guy that implemented it earlier) – this more or less looks like the Unreal 1 software renderer’s ‘bilinear filtering’ implementation,” Squarepusher said. “There is also a third-person chase cam view and a way to ‘lerp’ the animations (ie. add key-frame interpolation in order to make the animation of enemy models look smoother and have more frames of animation than they originally did).”

“I plan to eventually rebase the TyrQuake port and push it upstream to the original authors (ie. the maintainers of TyrQuake) – I did a lot of careless code rewriting that I’ll be sure to avoid for the rebase,” he added.

Shaders

There has been a major overhaul of the way shaders are handled, which has paved the way for advanced, multipass shaders that can be easily setup by end users without needing to tinker with any code. As part of these changes, the old XML/GLSL shaders with fixed-pipeline functions have been deprecated, but will still work just fine. In the future, we ask that interested shader authors try to stick to the multiplatform Cg format when possible. The GLSL/GLES formats will still be supported for compatibility with platforms that don’t support Nvidia’s Cg Toolkit, such as Android and iOS, and Cg shaders can be converted to these legacy formats programmatically using Themaister’s cg2glsl python script.

A couple of examples of newly written shaders that utilize some of the recently added features are Themaister’s NTSC Composite shader, which should work well on any libretro core, and Harlequin’s Gameboy shader:

There is no firm release date for v0.9.9, but if you would like to try any of these features out or get involved in the development, you can grab the code from git and compile it yourself for your platform of choice. If you have any questions about these features or RetroArch/libretro in general, stop by the libretro forums or drop by #retroarch on Freenode IRC.

Discuss this post