Why Lisp?

Lisp Logo
Lisp logo [Jooja, Wikimedia Commons]
Perhaps more specifically, why Lisp for creating a game? We’ve been working on Spycursion for a few years now. During all that time, at least once or twice, I have happened upon a video on YouTube with a title somewhere along the lines of “How I wrote a video game in 24 hours.” Each time the happy YouTuber leveraged a ready-to-go game engine such as Unity or Unreal. (There are even some that don’t start with U.) We are not using any of these. In a real sense we have taken it upon ourselves to reinvent the wheel. Why? It’s because we see value in creating our wheel using Lisp.

My first look at Lisp was during a course I took back in 1991 for my undergrad. I had a keen interest in artificial intelligence, but I had to drop the official AI course due to a scheduling conflict with one of my main degree requirements. As a consolation, I took what I somewhat dejectedly thought would be the next best thing. This was a 1-credit elective course on Common Lisp. I didn’t know much about Lisp beyond that it was said to be linked to AI. I was already working with Prolog at the time, which I then considered to be the “real” AI language. I felt I could make computing magic with Prolog. I could program in ways that went beyond what was possible using Pascal or C, the languages I needed for my other university courses.

The Lisp elective was very much my second choice, but I knew that Common Lisp was the language I would have been using in the AI course I had been forced to drop. To make my scheduling woes just a tiny bit worse, on the first day of class I got the impression that the instructor really wanted to be teaching Scheme, which she touted as a version of Lisp aimed at academic research. It was streamlined, slimmed down to just the essentials. Of course, to me it felt like Scheme would have been yet another jump away from the course I really wanted to take. The language didn’t even have Lisp in the name.

Even if it wasn’t my first choice, through that Common Lisp course I quickly learned to appreciate the power of Lisp. I surprised myself by writing a general problem solver for one of the earlier assignments. I gave it a fairly difficult problem to solve when I demonstrated it to the professor, realizing as I did that unfortunately my solver would likely need many years to come up with its solution. She still gave me an A for the assignment and ultimately for the course as well. I enjoyed that class thoroughly, even if at the time it wasn’t enough to make Lisp my go-to choice for AI. I still chose Prolog for my final senior design project. The Common Lisp course was, however, a critical first step on a path that has ultimately led me here to (defun games ()) and to Spycursion. I even took a quick look at Scheme a couple of times along the way.

When I finished my degree and went out into the world to work as a software developer, I more or less abandoned both Prolog and Lisp and made my living using “industry-standard” languages such as C, C++, and Java. Many years later, I returned to school for a master’s program in computer engineering. My studies and research were concentrated in the domain of machine learning. This is a specialized subfield of AI, but as I worked my way through the required courses, I thought I would finally have the opportunity to pick up that class on general AI, which those scheduling requirements had snapped away from me during my undergrad. I mostly succeeded. The graduate-level AI course was not offered, but the undergraduate version of the class was. My advisor, however, strongly recommended that I not take the class, advising me to only schedule graduate-level courses, even though the master’s curriculum allowed me one senior-level undergraduate class in my field of study.

I decided it would be wise to follow my advisor’s recommendation, but I was not going to let a second opportunity for this AI course slip away. I audited it, and indeed I loved it. However, there was one small problem, one which seemed immediately to rub me the wrong way. The examples and coding problems in the class would all be in C. Perhaps I should have been pleased because I had been working in C for many years and knew the language like the back of my hand. But it felt wrong. Lisp has played an interesting role in the history of AI, but the instructor didn’t really go over that history. He did, however, devote two classes to teaching basic Lisp programming. On the first of these two days, he told us that when people involved in the initial development of Lisp demonstrated to computer programmers versed in Fortran exactly what Lisp makes possible, these programmers responded with something along the lines of, “Wow! That’s slow!”

I wasn’t around back then; yet, I have no doubt that the initial Lisp interpreter from the late 1950s was slow compared to what those Fortran practitioners were used to. However, modern Lisps tend to do quite well in performance tests. C and Fortran may still be faster in terms of raw execution speed, but modern Lisps certainly hold their own. They also generally run faster than other popular languages like PHP and Python.

Of course, the wise developer knows that performance tests nearly always fall short when comparing languages, and any Lisp programmer will tell you that there is more to choosing a language than raw execution speed. For me, that sense of “more” is often linked to AI. Lisp quickly became the obvious answer for AI applications when this field was just getting started. Good old-fashioned AI is all about symbol manipulation. Lisp is a language for symbolic computation. The choice is clear (or rather it was throughout the 1960s and early 1970s).

However, Lisp is more than just a language for symbolic AI. It is a general-purpose language in a way that my beloved Prolog was never intended be. Lisp’s close ties to AI ultimately became a liability for the language. That clear choice in the early days became linked to projects for intelligent programs that worked inspiringly well on small simulations in a research lab, but which fell disappointingly flat when scaled up to handle real-world problems. Promises of our being just a decade or so away from machines that think like humans became empty. The problems were too big. The computers weren’t powerful enough. Funding dried up. The very term AI started to have negative connotations. As all this happened, Lisp fell out of favour.

This was unfortunate since, as it is a general-purpose language, the advantages that come with Lisp are in no way limited to research and development in AI. What are these general-purpose advantages? There are a good many that we could discuss, but one of the most important is that Lisp allows the developer to customize the language itself. This feature is rare as it is extremely difficult to implement in a non-Lispy language. Programming languages have strict syntactic rules that developers must follow explicitly if they want their programs to run. Lisp allows the programmer to add new rules or to tweak the existing rules in order to extend the language and to customize it for the application being developed. The power to mould the language itself gives an obvious advantage when attempting to handle the multitude of complex problems inherent in AI. Yet, complex problems also occur outside of the field of AI. In general programming, that same power is surprisingly handy whenever one sees the need to invent a better wheel. We want our wheel to take players to a better gaming world.

Of course, the term AI no longer carries the reputation of an empty promise. AI has made its comeback. This happened slowly, first with researchers essentially avoiding the term, labelling their work using the relevant subfield: expert systems; data mining; machine learning; and, most recently, deep learning. Today, people are not asking whether or not AI is possible (excepting an appreciable number of AI experts who focus on that very question). People are asking instead, when will it be here? Some are convinced that it is here already. A few are even concerned with important questions about safety as we quickly advance in our ability to create intelligent machines.

However, these days not too many are overly concerned about Lisp’s role in AI. As AI began to move forward again under various names, C, C++ and Java stepped in to replace Lisp. Today, Python seems to have become the new language of choice for AI development.

At (defun games ()) we still choose Lisp. And while there is a certain level of AI functionality incorporated into Spycursion (and while I thoroughly enjoy coding that functionality in Lisp), AI is not the reason. We are not looking to implement the 24-hour game. We want a better gaming wheel, and the advantages coming with Lisp give our tiny team a chance at successfully inventing one. We hope that you, our players, will be the ones who benefit. We are aiming to create a deeper and more complete gaming world for you to enjoy, a world we hope you will love getting lost in.

How Many Lisps Do We Need?

Lisp logo
Lisp logo by Conrad Barski

What? We’ve had blog posts about the fact that we’re writing Spycursion in Lisp. Some call it secret alien technology.  Okay, full disclosure: Lisp as a computing language was created on Earth by a human (his name was John McCarthy) back in the late 1950s. I can’t say whether or not he was inspired by beings from another world. I don’t know for sure, and even if I did, it would be a secret. I would never say. What I do know is that there is a great deal of power packed into the elegant and surprisingly simple programming language known as Lisp. Every few years, we see more modern languages (say, C++, Java, or Python) bringing in “novel” programming constructs into newer releases of the language. Yet, these exciting new constructs are frequently taken from Lisp and often have been part of Lisp since darn near the beginning.

So, it’s a great language if you’re a programmer willing to venture outside the proverbial box. But why did I start off talking about Lisps, plural, as if there were more than one? Simple: there is more than one. In fact there are many. Lisp is actually a family of programming languages. A Lisp can be quite different from other Lisps in much the same way that C++, Java, and Python are different from each other. (If you’re curious, those languages all belong to what could be called the Algol family.) What we have been generally referring to as Lisp is actually a language called Common Lisp, which is the result of an effort to consolidate features from many of the earlier Lisps into one standardized language. Common Lisp is a powerful choice, which allows our tiny team of developers to deliver you a great big gaming experience full of secrets, intrigue, and espionage in a dystopian world just waiting for you to leave your own unique mark on it.

But Spycursion is an MMO. For the three of you who don’t already know it, that means “massively multiplayer online” game. We would like you to know that we are taking that first M very seriously. No doubt you have heard about some online service that fell to its knees at some point because a mass of people all tried to connect to it at the same time. I remember, as a keen example, an evening in recent history when newscasters were reporting the increasingly certain result for a presidential election in the United States. So many people in the country were frightened by the outcome that the Canadian immigration website eventually went down that night. Some people thought this funny. Others thought it worrisome. I thought, Canada did not program its immigration website in Erlang.

So, what’s an Erlang? It’s not a Lisp (though that would certainly have been a sharp guess). It is, however, a programming language and an extremely powerful one at that, just not in the same way that Lisp is known to be powerful. Erlang was not designed to be a general purpose language with Lispy features that lets a tiny development team like ours mould the language itself into the perfect tool to build Spycursion for all of you. Erlang was designed specifically so that when hundreds of thousands (or dare we imagine, millions) of you all decide to play Spycursion at the same time, our servers don’t fall flat as if they had nothing more important to do than to provide information about immigration to Canada to panicking voters.

Erlang is a programming language whose central purpose is to do a massive number of tasks at the same time and also to create systems that never go down, ever. The downside of a language that strives to achieve what no other programming language can is that it is not general-purpose like Common Lisp. Erlang is primarily a server-side language, and so we aren’t really aiming to keep your own personal computer running. Rather, we want to make sure the computers responsible for providing you access to the world of Spycursion are able to keep that world available to you, even if you all decide to enter the game at the same time, even when we have to update our systems, even if we have hardware issues, and even if one of us has somehow missed a bug in our code. (There’s less chance of that in Erlang, but it still happens to the best of us.)

Yet, as we said, Erlang is not a Lisp. We are (defun games ()), which has Lisp right there in the name. (And we’ll give one month of free game play to the first 8 people who figure out how that makes any sense.) Sure the Spycursion client you enjoy on your computer is powered by Common Lisp, but we also want that Lispy goodness on the server side so we can fine-tune our programming language and bring you the best game possible. Unfortunately, there is almost always a trade-off when making design decisions about programming languages. If only Erlang were a Lisp so that we could harness the superpowers of both languages at the same time.

Now I am guessing that perhaps at most three of you have heard of LFE. It may be more of that secret alien technology, but it too was created on Earth by a human (his name is Robert Virding) back in the mid 2000s. LFE stands for Lisp Flavoured Erlang.  As the name implies, we have Lisp and Erlang in a single programming language. We at (defun games ()) feel that it truly represents the best of both worlds. We can use the power of a Lisp to extend the language so that it is not only a gaming language, but a language oriented specifically for Spycursion. At the same time we can build the world of Spycursion on the solidly reliable and massively parallel foundation that comes with an Erlang-based application.

LFE logo
LFE logo by Duncan McGreggor

So, how many Lisps do we need?   To provide you the best massively multiplayer online gaming experience we possibly can, we need two: Lisp Flavoured Erlang to serve you en masse and Common Lisp so we can delight you individually.

Introducing Myself (den)

Den's Picture
The peril-sensitive sunglasses need an adjustment!

Hello, Spycursion fans.  My name is Den Drown.  I have heard at various points that there are eight or three or a handful of you.  I had better introduce myself now, before you number in the hundreds or thousands.  (Dare I say millions?)  However many of you there are, know that I am thrilled to be part of the team bringing you Spycursion.  I know you’re simply going to love this game.

How can I know this when I’m just meeting you now, virtually, on a blog post for this odd company (defun games ())?  Essentially, I believe you’ll love the game for the same reason that I do.  The people bringing it to you are thinking of you at every step.  This is more of a challenge than you might imagine.  Some of you are interested in learning some programming.  Some just want to have fun.  Generally, you think a dystopian world, rife with espionage and subterfuge, sounds intriguing.  You are, of course, right about this, but your intrigue takes shape in various ways.  Some of you feel a deep desire to make this tragic world a better place.  Others feel they can understand the chaos, allowing them to manipulate it, bending it subtly to their will for fun and for profit.  Finally, our greatest challenge will surely come from those among you who seek some winding path between the noble goal and the devious one.  What are you looking for when you step into Spycursion?

No doubt, you’re already aware that (defun games ()) is coding Spycursion in Lisp.  Believe it or not (likely a good number of game developers will not), this is a very good thing.  My first experience with Lisp  was way back in 1989 when I took a course in Common Lisp as an elective in the computer engineering program at North Carolina State.  That was the semester I truly discovered the magic in programming.  My career in software development has largely revolved around the languages of the industry (C/C++, Java and several others of their persuasion), but over the last several years I have been fortunate enough to find (or create) opportunities to design and develop systems using a number of languages that allow the coder to ultimately go beyond what is readily doable using a “mainstream” programming language.  Most of these marvelous languages are members of the Lisp family.

Lisp is a key reason that I am joining Scott and Mauricio as we dedicate our blood, sweat, and tears to bring you Spycursion.  Lisp does more than make it possible for a tiny team like ours to create a professional game that will thrill you, delight you, and even allow you to learn something in the process.  John Foderaro called Lisp a “programmable programming language,” and it is exactly that.  We are molding the very language used to create Spycursion into the perfect language to express in code the world that the game represents.  To me, this beats the approach of plugging the game context into a mainstream gaming engine.  It beats it by a mile.

I may be biased.  Lisp has historically been considered the language of artificial intelligence, and here I am joining (defun games ()) just as I am working to finish up a PhD in cognitive computing.  However, if you are already a fan of Spycursion, well before even a beta release, then I bet you can guess how AI-oriented design and Lisp are a natural foundation for a game that challenges you to survive and even thrive in a world where technology is perhaps the only thing holding everything together.  In addition to AI, I will be drawing from a background in computer-telephony integration and information security.  I look forward to putting these skills to use so that you may lose yourself in the dangerous and captivating, technological world that is Spycursion.

–den Drown