While porting and rewriting ‘Craft’ for use as a libretro core, we decided it would be nice to document some of the steps involved for the purpose of education.
You can check out our source code repository here.
There might be addendums and followups to this article later on. Note that some of these steps are not things that are ‘required’ to be done for the purpose of porting software to the libretro API, they are simply best practices based on the subject matter at hand.
Step 1 – Getting it compiled
‘Craft’ uses Cmake as its build system. While libretro places no requirements on which build system you use for your project, usually out of habit and preference we prefer to write static Makefiles for convenience and portability instead.
We reuse a basic Makefile template for this that we import into other projects. The three files we will be creating are : Makefile.libretro, link.T, and Makefile.common. We will put these files into the root of the project (or any other place where the central Makefile is usually stored).
To make things easier to understand, Makefile.libretro is the general Makefile solution which includes Makefile.common. Makefile sets up all the platform targets that your core supports, while Makefile.common would be the equivalent of CMakeLists.txt. You define all the files here that will need to be compiled here.
A rundown on some of the variables inside Makefile.common :
SOURCES_C – You add C source code files to this variable.
SOURCES_CXX – You add C++ source code files to this variable.
INCFLAGS – You add include directories to this varaible.
Craft is a C-only program, so we will add all the source files to SOURCES_C.
First we will attempt to compile as many files as possible before we will move on to actually making it work with libretro. Later on, we might replace some of the dependencies that ‘Craft’ uses with some of our own for reasons of portability and consistency. Some dependencies we will avoid for now like glew and glfw, we will add our own substitutes for this later on.
You will notice after following this commit that there are still quite a number of undefined references left before the core will actually be assembled. That is because of some of the dependencies we have omitted so far (like glew and glfw), and because we have yet to write a libretro implementation.
The following are notes based on the dependencies found in Craft, if you don’t find any of these dependencies in another project you don’t need to be concerned about this.
NOTE: I defined SQLITE_OMIT_LOAD_EXTENSION and appended it to CFLAGS since we won’t be needing such functionality for the libretro port.
NOTE2: Although the developer of Craft has gone to great pains to make sure there are as few hard dependencies as possible (and the few there are, have their sources provided inside the project’s codebase) ,there are still some hard dependencies which will require us to pull in some sources later on, like ‘libcurl’. We will ignore this for now, and just dynamically link against it for now until we can get rid of this dependency by compiling it into the core itself.
Step 2 – Skeleton libretro application
We will add a skeleton libretro source file now to the project. We will add these files to the ‘src’ directory:
– libretro.c
– libretro.h
This will be the main program file of our libretro core. Unlike with normal programs, main() is not the default entry point. In a libretro core, there is no default entry point and instead you go through the libretro program lifecycle for initialization and deinitialization of a program. We will return to this later.
Most of the functions right now are purposeful stubs. We will start filling them in in later steps.
NOTE: Because ‘Craft’ is a program that requires no content to run (as in – you don’t need to provide it with say a ROM or an image file in order for it to start), we want the libretro core to start immediately after it has been loaded. Therefore we do a RETRO_ENVIRONMENT_SET_SUPPORT_NO_GAME call.
Step 3 – OpenGL context creation/state management
‘Craft’ uses several cross-platform OpenGL utility libraries in order to function. One is known as ‘glew’ (OpenGL Extension Wrangler Library), the other is called ‘GLFW’.
For libretro GL cores, we resort to several in-house libraries that can serve as replacements for these. For this port we will use two: glsym (equivalent to glew), and glsm (an OpenGL state machine).
We would like this port to work for both desktop OpenGL (2.0+) and mobile OpenGL ES (2.0 and up). Later on you will see that we will have to replace some functions like ‘glGetIntegerv’ inside the source code in order to meet this criteria. For now during this step we will focus solely on integrating glsm/glsym and setting up a working GL context.
We will add the following files to the project:
– deps/libretro-common/glsm/glsm.c
– deps/libretro-common/glsym/rglgen.c
– deps/libretro-common/glsym/glsym_gl.c (for desktop GL)
– deps/libretro-common/glsym/glsym_es2.c (for mobile GL)
In order to use the GL state machine, you first setup the GL state machine context inside the retro_load_game function. You can see this being done here:
A couple of notes on the second parameter we pass to glsm_ctl :
context_reset – this function callback is called after the GL context has been initialized. Usually you should try to clean up after yourself here, (re)initialize the graphics, reset the texture state, etc.
context_destroy – this function callback is called after the GL context has been destroyed. Usually you would destroy any allocated objects here related to GL or any other state.
stencil – set this to true if you require a stencil buffer.
The context_reset callback implementation needs some further explanation –
once we know that the GL context has been successfully created by the frontend, we then need to tell the GL state machine to reset its context and to set all default values for the state machine. We can do this by doing the following function calls:
This ends the context-related setup part of glsm. From there, the way you use it to simply call this at the start of each retro_run call –
and unbind the state again at the end of that same function call, like this –
Step 4 – Reimplementing the runloop
A regular program’s flow of control usually starts at the ‘main’ function. ‘Craft’ is no different. This function is defined inside the main.c file. From there, it sets up an infinite while loop that runs as long as the game keeps running. Once we exit the game, it will finally exit the function returning a value, signalling program exit.
To convert a program like ‘Craft’ over to the libretro API, we need to make several modifications so that the libretro application lifecycle is correctly implemented. We will go over some of the required functions to implement, and what to implement inside these callback functions.
To make Craft work as a libretro core, we have had to rewrite the entire main.c file and in specific the main() function. Instead of one long big function, we have had to create a couple of functions which will get called from our libretro function callbacks:
– void main_init(void);
– void main_deinit(void);
– void main_load_game(int argc, char *argv);
– void main_unload_game(void);
– void main_run(void);
– retro_init
This is the main starting point of a libretro core. You can initialize resources here that are needed for the program to run.
We will call ‘main_init’ from this function.
– retro_load_game
This is the function that gets called by the libretro frontend when the frontend passes content to the core and the core needs to do something with it. The retro_load_game function is always called no matter if we are starting the core with content or not.
In the case of ‘Craft’, we do not require any content to be loaded for the program to run. Therefore, we can mostly ignore the parameter ‘info’ passed to this function.
Because ‘Craft’ is a libretro GL core, we have to be aware of something: you can see that we early-return true inside this function. What will happen then is that GL context creation gets set up behind the scenes. Once the GL context is ready to be used, the function callback ‘context_reset’ gets called. Once this happens, we know that our GL context is ready to be used by the core.
This is why we have delayed the execution of ‘main_load_game’ instead of simply calling it inside this function. Instead, we will call ‘main_load_game’ from retro_run when we know the GL context is ready.
– retro_run
This is the libretro core’s equivalent of an infinite while loop, with the exception that retro_run is supposed to represent one video frame’s worth of execution. By default, we assume that the frame time will be equal to what you need for 60 frames per second gameplay, although it is possible to configure the frame time on a per-retro_run basis with libretro as well. This core will not require us to toy with this, so we ignore this aspect for now.
In a libretro core, it is assumed that you need to call the video_cb callback at least once for every retro_run iteration, same with the audio callback. Since “Craft’ doesn’t have any audio code, we won’t bother with this.
As we mentioned earlier during our explanation of ‘retro_load_game’, the initialization of the GL hardware context might take some time until it’s ready. Until then, we cannot use any GL functions. Once it’s ready, the ‘context_reset’ callback will set the static variable ‘init_program_now’ to true. We check for this inside retro_run :
if (init_program_now)
{
main_load_game(0, NULL);
init_program_now = false;
video_cb(NULL, game_width, game_height, 0);
return;
}
As you can see, we call ‘main_load_game’ at this point to finally initialize the game. You will also notice that we set the init_program_now variable to false, so that this function will no longer be triggered. We then do an early return from the function and emit a duped frame. We are now ready to be rendering the game in subsequent iterations of retro_run.
At this point, the main execution flow in subsequent calls of retro_run will be as follows:
– bind the GL state machine
glsm_ctl(GLSM_CTL_STATE_BIND, NULL);
– poll input
input_poll_cb();
– optionally process input
– call main_run so that we iterate the Craft core for one frame.
if (main_run() != 1)
{ }– unbind the GL state machine
glsm_ctl(GLSM_CTL_STATE_UNBIND, NULL);
– render to screen
video_cb(RETRO_HW_FRAME_BUFFER_VALID, game_width, game_height, 0);
– retro_unload_game
It will call this function upon shutting down the core. In here, we deinitialize/free
any resources directly related to the ‘game’ content we had loaded.
For ‘Craft’, no game content has actually been loaded since the core does not
require external content in order to run.
We will call main_unload_game from this function.
– retro_deinit
This function is called after ‘retro_unload_game’. In here, we will deinitialize all
other resources related to the core itself that weren’t already deinitialized during
‘retro_unload_game’.
We will call ‘main_deinit’ from this function.
Step 5 – Baking in assets
The original ‘Craft’ will compile its shaders from source files stored inside a subdirectory, and it will also look up its textures inside a subdirectory.
Since these shaders are pretty much required for the program to work and because the textures are relatively small, we made the decision to bake these assets into the main code. That way, we will not burden the user with having to store these assets somewhere where the libretro core can find and use them. We can also later on make changes to them easier for the purpose of OpenGL ES compatibility.
To convert the PNG files, we will use a program called bin2c (which can be found on Github), and perform the following commands:
bin2c textures/font.png textures/font_texture.h font_texture
bin2c textures/sign.png textures/sign_texture.h sign_texture
bin2c textures/sky.png textures/sky_texture.h sky_texture
bin2c textures/texture.png textures/tiles_texture.h tiles_texture
We now include these headers inside the src/main.c file.
Craft uses a middleware tool called ‘lodepng’ in order to load and decode PNG files into memory. The original code would use lodepng’s file I/O loading function in order to load PNG files. Since we are going to be using PNG files from memory instead, we will need to use a different function provided by lodepng: lodepng_decode32. This will allow us to point it to a source memory buffer (the converted in-memory PNG file) and then have it dump its decoded output to a destination memory buffer, which we can then provide to OpenGL.
The code we arrived at ended up looking like this:
static void load_png_texture_data(const unsigned char *in_data, size_t in_size)
{
unsigned int error;
unsigned char *data;
unsigned int width, height;
error = lodepng_decode32(&data, &width, &height, in_data, in_size);
if (error) {
fprintf(stderr, “error %u: %s\n”, error, lodepng_error_text(error));
}
flip_image_vertical(data, width, height);
#if defined(HAVE_OPENGL) || defined(HAVE_OPENGLES)
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA,
GL_UNSIGNED_BYTE, data);
#endif
free(data);
}
We then call it from the function ‘upload_texture_data’ for each texture file, which in turn gets called like this:
upload_texture_data((const unsigned char*)&tiles_texture[0], tiles_texture_length, &info.texture, 0);
upload_texture_data((const unsigned char*)&font_texture[0], font_texture_length, &info.font, 1);
upload_texture_data((const unsigned char*)&sky_texture[0], sky_texture_length, &info.sky, 2);
upload_texture_data((const unsigned char*)&sign_texture[0], sign_texture_length, &info.sign, 3);
In followup articles, we might touch on the process of baking in the GLSL shaders required for this core to function, and backporting these shaders to GLSL 1.00 so that they can be used with OpenGL ES 2 (a current work-in-progress). It might also touch upon what was needed in order to get keyboard and mouse functionality to work with the libretro core, and some of the other development changes we made to the core so far.
Iris Johnson
June 7, 2016 — 1:17 pm
You didn’t touch on this in the article, but given that a OpenGL 2.0 target doesn’t have the same features as an OpenGL ES 2.0+ target, would you be able to target later OpenGL features if supported in the hardware?