Why Lisp?

Lisp Logo
Lisp logo [Jooja, Wikimedia Commons]
Perhaps more specifically, why Lisp for creating a game? We’ve been working on Spycursion for a few years now. During all that time, at least once or twice, I have happened upon a video on YouTube with a title somewhere along the lines of “How I wrote a video game in 24 hours.” Each time the happy YouTuber leveraged a ready-to-go game engine such as Unity or Unreal. (There are even some that don’t start with U.) We are not using any of these. In a real sense we have taken it upon ourselves to reinvent the wheel. Why? It’s because we see value in creating our wheel using Lisp.

My first look at Lisp was during a course I took back in 1991 for my undergrad. I had a keen interest in artificial intelligence, but I had to drop the official AI course due to a scheduling conflict with one of my main degree requirements. As a consolation, I took what I somewhat dejectedly thought would be the next best thing. This was a 1-credit elective course on Common Lisp. I didn’t know much about Lisp beyond that it was said to be linked to AI. I was already working with Prolog at the time, which I then considered to be the “real” AI language. I felt I could make computing magic with Prolog. I could program in ways that went beyond what was possible using Pascal or C, the languages I needed for my other university courses.

The Lisp elective was very much my second choice, but I knew that Common Lisp was the language I would have been using in the AI course I had been forced to drop. To make my scheduling woes just a tiny bit worse, on the first day of class I got the impression that the instructor really wanted to be teaching Scheme, which she touted as a version of Lisp aimed at academic research. It was streamlined, slimmed down to just the essentials. Of course, to me it felt like Scheme would have been yet another jump away from the course I really wanted to take. The language didn’t even have Lisp in the name.

Even if it wasn’t my first choice, through that Common Lisp course I quickly learned to appreciate the power of Lisp. I surprised myself by writing a general problem solver for one of the earlier assignments. I gave it a fairly difficult problem to solve when I demonstrated it to the professor, realizing as I did that unfortunately my solver would likely need many years to come up with its solution. She still gave me an A for the assignment and ultimately for the course as well. I enjoyed that class thoroughly, even if at the time it wasn’t enough to make Lisp my go-to choice for AI. I still chose Prolog for my final senior design project. The Common Lisp course was, however, a critical first step on a path that has ultimately led me here to (defun games ()) and to Spycursion. I even took a quick look at Scheme a couple of times along the way.

When I finished my degree and went out into the world to work as a software developer, I more or less abandoned both Prolog and Lisp and made my living using “industry-standard” languages such as C, C++, and Java. Many years later, I returned to school for a master’s program in computer engineering. My studies and research were concentrated in the domain of machine learning. This is a specialized subfield of AI, but as I worked my way through the required courses, I thought I would finally have the opportunity to pick up that class on general AI, which those scheduling requirements had snapped away from me during my undergrad. I mostly succeeded. The graduate-level AI course was not offered, but the undergraduate version of the class was. My advisor, however, strongly recommended that I not take the class, advising me to only schedule graduate-level courses, even though the master’s curriculum allowed me one senior-level undergraduate class in my field of study.

I decided it would be wise to follow my advisor’s recommendation, but I was not going to let a second opportunity for this AI course slip away. I audited it, and indeed I loved it. However, there was one small problem, one which seemed immediately to rub me the wrong way. The examples and coding problems in the class would all be in C. Perhaps I should have been pleased because I had been working in C for many years and knew the language like the back of my hand. But it felt wrong. Lisp has played an interesting role in the history of AI, but the instructor didn’t really go over that history. He did, however, devote two classes to teaching basic Lisp programming. On the first of these two days, he told us that when people involved in the initial development of Lisp demonstrated to computer programmers versed in Fortran exactly what Lisp makes possible, these programmers responded with something along the lines of, “Wow! That’s slow!”

I wasn’t around back then; yet, I have no doubt that the initial Lisp interpreter from the late 1950s was slow compared to what those Fortran practitioners were used to. However, modern Lisps tend to do quite well in performance tests. C and Fortran may still be faster in terms of raw execution speed, but modern Lisps certainly hold their own. They also generally run faster than other popular languages like PHP and Python.

Of course, the wise developer knows that performance tests nearly always fall short when comparing languages, and any Lisp programmer will tell you that there is more to choosing a language than raw execution speed. For me, that sense of “more” is often linked to AI. Lisp quickly became the obvious answer for AI applications when this field was just getting started. Good old-fashioned AI is all about symbol manipulation. Lisp is a language for symbolic computation. The choice is clear (or rather it was throughout the 1960s and early 1970s).

However, Lisp is more than just a language for symbolic AI. It is a general-purpose language in a way that my beloved Prolog was never intended be. Lisp’s close ties to AI ultimately became a liability for the language. That clear choice in the early days became linked to projects for intelligent programs that worked inspiringly well on small simulations in a research lab, but which fell disappointingly flat when scaled up to handle real-world problems. Promises of our being just a decade or so away from machines that think like humans became empty. The problems were too big. The computers weren’t powerful enough. Funding dried up. The very term AI started to have negative connotations. As all this happened, Lisp fell out of favour.

This was unfortunate since, as it is a general-purpose language, the advantages that come with Lisp are in no way limited to research and development in AI. What are these general-purpose advantages? There are a good many that we could discuss, but one of the most important is that Lisp allows the developer to customize the language itself. This feature is rare as it is extremely difficult to implement in a non-Lispy language. Programming languages have strict syntactic rules that developers must follow explicitly if they want their programs to run. Lisp allows the programmer to add new rules or to tweak the existing rules in order to extend the language and to customize it for the application being developed. The power to mould the language itself gives an obvious advantage when attempting to handle the multitude of complex problems inherent in AI. Yet, complex problems also occur outside of the field of AI. In general programming, that same power is surprisingly handy whenever one sees the need to invent a better wheel. We want our wheel to take players to a better gaming world.

Of course, the term AI no longer carries the reputation of an empty promise. AI has made its comeback. This happened slowly, first with researchers essentially avoiding the term, labelling their work using the relevant subfield: expert systems; data mining; machine learning; and, most recently, deep learning. Today, people are not asking whether or not AI is possible (excepting an appreciable number of AI experts who focus on that very question). People are asking instead, when will it be here? Some are convinced that it is here already. A few are even concerned with important questions about safety as we quickly advance in our ability to create intelligent machines.

However, these days not too many are overly concerned about Lisp’s role in AI. As AI began to move forward again under various names, C, C++ and Java stepped in to replace Lisp. Today, Python seems to have become the new language of choice for AI development.

At (defun games ()) we still choose Lisp. And while there is a certain level of AI functionality incorporated into Spycursion (and while I thoroughly enjoy coding that functionality in Lisp), AI is not the reason. We are not looking to implement the 24-hour game. We want a better gaming wheel, and the advantages coming with Lisp give our tiny team a chance at successfully inventing one. We hope that you, our players, will be the ones who benefit. We are aiming to create a deeper and more complete gaming world for you to enjoy, a world we hope you will love getting lost in.