Hi, Mauricio Here,

This blog post is kind of a developers blog, as I said in my only previous post I’ve been dealing with the MS Windows side of things, I also made some 3D models using Blender, and use the abstractions Scott had created to create a a new scene for the game (a bedroom), but of course my job has not been limited to that. We’ve learned some stuff that we would like to share, so that is what I am doing.

Classimp on Windows.

One of the earlier lessons we learned was related to Assimp, for one thing downloading the binaries and moving the dll to your project directory does not work, in recent Assimp versions the name of the dll isn’t even one of the names searched by Classimp, and renaming or making the code match the dll name does not do the trick. In order to make classimp work we had to compile with gcc instead of visual c, using mingw,, this produces a libassimp.dll which is what classimp looks for and that DLL works fine. The latest Assimp library supported by Classimp is 3.3.1, so if you want to compile it yourself download that version and then follow the instructions here.

When you compile with MingGW using the instructions in the link the DLL will be in the “assimp-3.3.1\build\tools\assimp_cmd” directory, and asides from that you get an assimp.exe executable that is very useful for asset conversion. Currently we are consuming fbx files, so when I created models in Blender I exported them as fbx files, and that caused problems in some cases, so we adopted a different  workflow which so far guarantees a model that loads correctly in CEPL, we use assimp.exe in the command line to convert the .blend to .dae which is the extension for Collada, an open exchange format, and then use Autodesk FBX Converter to convert the .dae to .fbx. This workflow gives you better control over the way that the files are converted.

Why not reading the .blend files directly?, Well for one thing we had some pre-made assets and those were fbx, and we want to be uniform., another reason is that sometimes the file would not be read correctly by classimp, but assimp.exe tries several stuff and is able to export it, as long as you remove any lamps form the scene (I also remove the cameras for good measure). And a third reason, is that sometimes we want to pre-transform the vertices  specially with several objects in the same file, and doing this during the export simplifies the code to read the file. So we do use this:

assimp export file.blend  file.dae -ptv

In the command line, where -ptv stands for pre-transform vertices, and assimp guesses the output file format from the extension of the second file. Assimp does not export fbx , that is why we export to ,dae and then convert. Classimp can read .dae files correctly, but fbx files are smaller, and the difference in load time is negligible for our purposes, and as I said we want to keep uniformity.

Another good tip is to use the assimp viewer to open an asset, and then look at the log file, this will warn you of possible problems when importing with classimp.

Multiple Textures in one Mesh.

We ran into a problem with a model of an avatar we were using in the game, the textures were not imported correctly, the head texture was applied to the body, so the model looked like it had no pants, and it had some weird things in the skin (actually stretched head features). The problem is that when you run (classimp:import-into-lisp) each mesh object contains only one texture. however the model had a mesh with two textures.

Luckily Blender allows you to select the portion of the mesh that is assigned to a particular texture and separate it. So we separated the head from the body, re exported, re-sorted the textures and voila, problem solved.

How do you know the order of the meshes? Well, the first thing is to name things right in Blender, then the structure you get when you import into lisp has a tree and that gives you the clue.

SPYCURSION-CLIENT> (ai:import-into-lisp (car *char8-data*))
#<CLASSIMP:SCENE {100A0E4AE3}>
SPYCURSION-CLIENT> (ai:root-node *)
#<CLASSIMP:NODE {1004FA9443}>
SPYCURSION-CLIENT> (ai:children *)
#(#<CLASSIMP:NODE {1004FA8663}> #<CLASSIMP:NODE {1004FA8903}>
  #<CLASSIMP:NODE {1004FA8B93}> #<CLASSIMP:NODE {1004FA8E33}>
  #<CLASSIMP:NODE {1004FA90D3}> #<CLASSIMP:NODE {1004FA9373}>)
SPYCURSION-CLIENT> (loop for x across * collect (ai:name x))
("Head" "Gadget" "Bag" "Body" "Jacket" "Hair")
SPYCURSION-CLIENT>

So the first mesh is the head, the second the Gadget, and so on. The pre existing textures worked well because I just separated and saved again so the UV coordinates remain intact,  now each mesh reads from the right image, and maps the right coordinates.

Blender

One of the models I was making was a full room, to do it I used Archimesh, the room was simple, a rectangular room with a window and a door,  and some cabinets, this worked out perfectly in Blender, but not so much when I imported the models in CEPL. The thing is that boolean operators produce weird geometry, and the holes in the walls for the door and window are produced by boolean subtraction. In other models the sub surface modifiers looked like crap when imported,

So I decided to re-make the room the old fashioned way, loop-cuts and face deletion, replace the old room, and apply all of the modifiers to other objects before exporting, that way the look was predictable.

The other thing I learned is that you need to either use the a processing flag in classimp or export the model with pre-transformed vertices, otherwise the objects in the scene were not placed in the right position. I used Blender to lay out the room furniture, and then exported each element separately, that made it easier to place and remove things, and handle the textures.

I also spend some time baking textures, and exporting the UVs, the UVs exported from Blender are immediately usable, and you can use DIRT to import the texture, so asides from the tedious process of UV wrapping the objects, assigning the image, and baking the textures into it, the process is very painless.

Final Thoughts

After ironing out the exporting problems saving models so they can be used with CEPL turns out to be a straight away process, there are some things in our code we are optimizing to make the process of maintaining the software and adding stuff more straight forward, but we have a working skeleton,  we can create rooms and cities, place avatars, animate them,  etc.

Of course there is still work to do, and Spycursion is curious in that it has two environments, the OS and the 3D world, but even when we some times hit ourselves against a wall, there is always a door a couple of steps away, as we learn more is like turning the lights on, we now where the walls doors, windows and other stuff is, and that allows us to move freely.

Lisp programmers are a small group. According to the TIOBE index, Lisp currently sits at #32 on their ranking of programming language popularity. (Common Lisp specifically is somewhere between 51 and 100.) You can certainly imagine that Lisp game programmers are an even smaller group. And then when you consider non-trivial games, the games made by teams, the ones that are significant enough in scope to potentially fund the studio building them… well, I don’t think it’s much of a leap to say that you can probably count the number of those games on one hand.

Suffice it to say that not a whole lot has been written on the art of developing games in Lisp, relative to other languages. As a leader of one of the aforementioned five-or-fewer teams building a sizeable Lisp game (Common Lisp, in our case), I want to contribute to that collective knowledge base. But first, a caveat: It would be premature to consider either (defun games ()), as a studio, or myself, personally, as any kind of “authoritative voice” in the Lisp game community. Spycursion is the first game that we as a studio are producing, and it hasn’t even been funded yet. (Hint hint, Kickstarter, Feb. 26, hint hint!) And although I was a hobbyist Common Lisp developer for about seven years before starting work on Spycursion last year, there’s still a lot about this language I don’t know, and there are certainly people who could write better/faster Lisp game code than me… I’m just the only one who was insane enough to start this project. 😉 We also recently picked up another developer (he hasn’t introduced himself yet, but he will) whose opinions may differ from mine. Taken altogether, what I’m trying to say is: I’m not a guru. I’m just one guy trying to find his way on this bizarre-yet-beautiful landscape, just like everyone else!

Aaaanyway, Lisp games. The first thing you should know, before you go tell all your friends “Hey, they wrote Spycursion in Lisp, let’s create our Overwatch/Pokemon/Dark Souls cross-over in Lisp, too!” is that Spycursion is weird. The game itself is weird, its architecture is weird, its developers are weird… Point is, we’re working with some parameters that make Lisp not only suitable for Spycursion, but perhaps even the ideal language for it. Your Overwatch/Pokemon/Dark Souls cross-over may not fit the bill. (Or it may. I’ll let you know after we release Game #2.) It’s also important to remember that Spycursion has a client and a server component, both of which are written in CL. This was not always the case for the client; I’ll share that story in a minute. But let’s talk about the server first.

As we’ve mentioned before, Spycursion has its own programming language. You may have read about how domain-specific languages (DSL’s) in Lisp are grand, how they’ll make your coffee for you, blah blah blah. In my experience, a lot of the conversation about this can get pretty academic, but here’s a summary of what happens when you compile and run a Slang program in the world of Spycursion:

1. Compilation: The server parses your program (assuming it is syntactically correct) and creates a tree of Lisp forms from it.
2. Runtime: Some contextualization happens with that tree of Lisp forms, and then it is evaluated within the context of an in-game device.
3. (Optional): If the program needs to communicate with another program on another in-game device, via in-game network, then some magic happens to determine where that program is, and runs it, too.

This is of course a lot more complicated than I’m making it sound, but it’s still far less complicated than in probably most, if not all, other mainstream languages. Lisp allows you, as a player, to run your own code within someone else’s online game. (And yes, we do run security checks on your code before we parse it. We’re not n00bs, thankyouverymuch.) This is not merely simulated. There are plenty of places within Spycursion where we “cheat” to make things easier on ourselves — hey, we’re an indie, cut us some slack — but Slang is not one of those places. Just thinking about trying to do all of this in C++ gives me an icky feeling. (I eagerly await the e-mail from someone telling me how you can do it in C++, if you just jump through these 27 hoops…)

So, yes, Lisp is awesome as a game server, but I have a feeling that’s not what most of you are here for. You want to know why we’re using Lisp for the Spycursion client, and how you can write your own amazing thing in Lisp too, and clearly the answers are because I’m insane and you should stop listening to me because it’s fun, dadgum it, and by being a masochist early adopter and maybe having a fairly simple game, graphically. Sarcasm aside, these questions deserve longer answers:

Why Lisp (for the client)? Okay, story time. Long, long ago, when the Spycursion server was just Slang and a WebSocket interface and not much else, I set about creating a text-only testing client. I did this in Python, figuring it would be relatively quick and easy, which it was. At this point I’d not completely decided on how the client would be implemented, but I reasoned that, despite Python not being my best language, it would be easy to find developers for — so I could handle doing all the server-side Lisp as long as I had someone else dedicated to the client.

But there was a problem: Writing in Python just wasn’t fun. Some of you may not understand this, but it’s a phenomenon I’ve heard about before — once you get “spoiled” by Lisp, you have trouble getting motivated to code in any other “lesser” language. This was before we existed as a studio, when I was doing this project entirely by myself, so I didn’t have anyone else to shove the Python work off to, but I knew it was important to get something visual produced so that I could begin to show people (not simply tell them) what Spycursion was about.

So I chucked the Python client, rewrote the thing in CL, and around that time, started getting ideas of what I felt this fledgling studio could be. In my opinion, the reasons “nobody programs in Lisp” mostly boil down to, well… because “nobody programs in Lisp.” The community is small, so there are few tools available, so the community doesn’t grow because people don’t use Lisp for major projects due to a lack of tools. This is even more true for Lisp games specifically, but I believe that this chicken-and-egg problem can be helped by creating a well-known game in Lisp, if only to point to it as a success and prove that it can be done. Additionally, by open-sourcing the client code (which we’ve not done yet, mostly due to legal questions that need answering), we show new and aspiring Lispers how it can be done. Again, to avoid getting ahead of ourselves — this thing could still fail, perhaps in spectacular fashion. But at least if it does, you’ll get a front-row seat. 😉 And I guarantee you that it won’t be because of the programming language we chose.

Why do you think it will work? As I mentioned before, Spycursion is weird. There are very experienced people within the Lisp community who think that Lisp, in its current state, is not ready for serious game programming. I have no basis on which to disagree with them, but I do think it depends on the type of game you’re creating. Spycursion has several things going for it that make it different from what you might imagine as a “typical game,” especially a AAA game:

• There’s no twitch action. An ill-timed garbage collection isn’t going to be the difference between life and death. As of yet, we’ve made no attempt to customize GC or really push for low-level performance in ways that might be challenging to do in Lisp. We might, at some point, but I’m guessing we won’t even need to.
• Much of the tricky stuff is offloaded to the server. Don’t underestimate the advantage of having a server (read: one single platform, of your choice) running parts of your game. That means fewer shared library headaches and fewer client performance woes — assuming you do everything asynchronously — for starters. (Oh, but please don’t be that developer who forces a network connection for your single-player game. That’s just rude.)
• We don’t need to build for consoles. Ever tried to run Lisp on the Switch? I haven’t… and I don’t want to.
• Physics? We don’t need no stinkin’ physics. Or at least no physics that can’t be calculated based solely on what I remember from 9th grade Geometry.
• We keep graphics simple. I don’t really know how to quantify that, but the game is meant to be played 3rd-person, from a distanced zoom level. (We let you zoom in, but I wouldn’t necessarily recommend it.) The assets will be relatively low-poly and there will be relatively few of them. Note that “simple” does not mean ugly. We’re aiming to make it all look good while still putting most of our effort into gameplay — graphics are one of the places we “cheat.” You’ll be able to judge how we’re doing once we finish the Kickstarter trailer.

Put all of the above together, and you get a picture of a fairly minimalistic game client that really gets out of the way and lets you enjoy playing. (Or at least that’s the picture I hope you get!) And to be clear, I am not saying that fancier, AAA-level games can’t be made with Lisp. I’m saying I really don’t know, but this is our attempt to get as close as we can with the resources we have.

How are you doing it? I would say “pain, sweat, and toil,” but that kind of describes all game development, so…

• The engine. Oh, just kidding, we aren’t using one. In 2018, I know of at least three Common Lisp 3D game engines in varying stages of development: First Light, Trial, and cl-bodge. In 2017, when I was researching these engines, they were all either too early in development or otherwise just didn’t fit our needs. They might now, but I haven’t followed their progress enough to know for sure. This brings me to what is maybe the most important thing to know if you’re looking to write a non-trivial Lisp game in 2018: You have to be okay with writing a lot of things yourself. Lisp makes this fun to do, and might still even save you time overall depending on just how much you need to write. But it’s a very different mindset from being, say, a plug-and-play JavaScript developer.
• CEPL. CEPL is an abstraction layer around OpenGL which has been a joy to work with, once I figured out how. (Tip: See the author’s video stream, it’s great.) Its goal seems to be to make OpenGL lispy and fun to work with, and I really think it has achieved that. Spycursion uses CEPL extensively, and while I’m sure we’re doing some things suboptimally, performance has been fine so far. We’re also using cepl.sdl2 which brings in cl-sdl2.
• QTools. QTools is a lispy wrapper around Qt 4. Most games probably won’t want the heft of Qt, but our device UI actually needs something a bit more featureful, so it’s worth mentioning. We have run into some build problems with QTools, particularly on Linux. Add in that there doesn’t seem to be a Qt5 update on the horizon, and we may well end up switching away from it (to what, I can’t say yet). But it’s currently working well, for what we need it for.
• Other libraries
  • Assets: classimp, dirt
  • Fonts: cl-fond
  • Input: skitter
  • Network: websocket-driver-client
  • … and some other less-interesting stuff.

And that’s it! Lisp game development isn’t for everyone, but I stand by my assertion that it can be done, and done well, given the right game — and perhaps the right amount of patience.

TL;DR:

• Use CEPL. Or one of the engines, if they fit. Or maybe both.
• Yes, you should do it, even if you fail. That’s how the community grows.
• Be prepared to write a lot of stuff yourself. This is not necessarily a bad thing. It’s fun!
  • Except skeletal animation. Don’t write that yourself.
• Support our Kickstarter starting on February 26th!