Common Lisp: How deep does the rabbit hole go?
Being that both Scott and I had to go back to getting jobs to put bread in the table things are going slower than when we were both 100% dedicated to the project. In the last update Scott mentioned some ideas we had around re-structuring the client, as you may imagine we can manage to keep the server side purely in Common Lisp, but the client side is a bit different, we need to create UI elements, and 3D graphics, as well as adding assets. We talked about moving away from Qt, not because Qtools are not capable, but mainly because in it’s current state UI elements cannot be rendered in the OpenGL window, which creates a disjointed experience for the user.
So we started an internal discussion to evaluate the future of Spycursion’s development, things that we wanted, things that we needed contrasted to what we have and what we need to implement, and how home grown that implementation needs to be, of course we want to use existing solutions where it makes sense, but we have our own philosophy around development.
Quick story, we considered going down the path of using Godot, using gdnative to be able to program all of the game logic in Common Lisp while using the game engine capabilities to import assets, create the world, apply physics, etc. I even started the task of creating the bindings1 using CFFI. All of this seemed like a good contribution to the lisp game developers community, and there are plenty of Godot tutorials on line. So asides from having to redo everything in Godot, what stopped us?, it is not Lisp far enough down.
If we make it so we can program Godot’s game logic in Common Lisp we are only layering a coat of usability on top of an otherwise C++ stack. Lisp Hackers could only contribute to the engine through plug ins or functions that work with the API, but if they want to go deeper they have to switch languages. So the fair question now is, how far down we can go in our current stack before we have to switch languages?, after all a lot of what we do in game development is done through FFI. The answer is, arguably, farther down than if we just wrap Godot, of course with some trade-offs. So what will Godot do for us, here is a non comprehensive list:
- Asset management, sounds, music, sprites/models.
- A visual environment to create levels and assign properties to in-game objects (walls, ground, etc).
- A physics engine.
- Custom Shaders (with their own language).
- GUI which can be layered on top of the game.
- A simple scripting language for the logic.
What we have now with our prototype client:
- Asset management, mostly 3D models only, character animations are rendered correctly, 2D sprites can be imported as planes with texture.
- A non visual environment to create complex scenes.
- Custom Shaders using the excellent CEPL library created by Chris Bagley (akaBaggers), shaders are written in Vari which is very Lispy.
- The complete Common Lisp language for the logic.
The Missig parts
Asset management
We have not added any sound to our game yet, so we haven’t dealt with importing and managing sound including music, however there are several options out there like Pavel Korolev’s (aka Borodust) bodge-openal, or Nicholas Haffner’s (aka Shinmera) harmony, and some others. So we have native options we can integrate in our client with relative ease.
Scene management
Although I listed the “non visual environment” I guess I should mention two things. First, in our current work flow I was able to create a scene in Blender directly, and then import the elements one by one in the SC client replicating the placement from blender automatically. So in a way we have a visual environment to create the scene. I think however, that using CEPL it should be simple enough to create a solution where you move and rotate things around as needed. So far we haven’t got a need for that.
UI
We are working currently working on integrating bodge-ui (suggested by our good friend Borodust), which will allow us to layer UI elements on top of the game, this is what will substitute Qtools, I’ve made some basic testing and the UI renders well in Linux and Windows, I can’t check Mac but we will get around to testing more thoroughly once we integrate things in the Spycursion client. We also talked about having the users be able to create “web pages”, the solution in Godot seems to be using a combination of bbcode and the GUI, which got us thinking, we don’t really need a full blown HTML render in game, in fact is safer to use something like bbcode but with the addition of forms.
Is there need for a Common-Lisp game engine?
We have been constructing a game, not a game engine, using loose parts created by great programmers, by programming the code to be modular and reusable we get a similar effect to having a full blown engine, as an example. if we need to add another scene we just create a scene file, place the assets in a convenient place, and declare things and we have a scene ready to be rendered. I talked about Shinmera’s harmony, and Borodust’s bodge-ui, these wonderful programmers are already giving us the building blocks, so you can make your own mix and match, or you can choose to use their full solutions i.e. Radiance, CL-Bodge, etc. We even thought about going 2D with Dave O’Toole’s (aka dto) Xelf.
Ongoing work
There is a lot going, very slowly but going, we are refactoring the client to make the code modular an easy to reuse, some manual steps are being moved to CLOS methods (before, after and around), some hard coded values are being moved to special variables, etc. The code has to be easy to maintain, and our initial approach needed some refining. I’ve been working mainly on the client, and Scott more on the server, and a lot of what we have been doing is “invisible”, but we are doing what must be done. Hopefully as we add/refine the visual aspects of the game we will be able show the progress instead of just telling you about it.
Patreon
We want to publicly say thanks to our Patreon supporters, although we cannot live from their contribution some of the game expenses will be alleviated, and it shows us that there is still interest on having this game become a reality.