Posted on

What’s the deal with this server side and client side stuff?

I was looking at our previous posts and I noticed that Scott had written a bit of an overview of what the server does, but I never made any attempts to make any explanations about the client, I guess that somehow I tend to think that everybody knows, but some times is better to talk about what you think everybody knows. The client, is, after all, the most visible part of the game.

If you write a single player game, normally you can get away with having everything in a single place (the executing machine). The natural question when you have an online multiplayer game is, What can you have in your client and what must be on the server?

Let’s imagine a simple game which can be played online, “Air Hockey”, and of course, keeping it simple, just two players. Each client could have a map (the table), the assets (the paddles and puck graphics), the sound effects and music. Since each player is using a different computer to render the table, paddles, puck etc,  we could quickly come up with two versions of the world, the one you see and the one your opponent sees, and that can greatly affect game-play. If you try to block the puck but it is not where you thought it should be, then you could unfairly loose a point.

Right away we understand why we need a system to make our worlds as similar as possible. So both clients must communicate with the server, and tell it where the paddle is, what is its speed, and perhaps some other information, the server in this case will communicate to both clients where the puck is, what is its speed and direction, where is your opponent’s paddle, and how it is moving, state that tells you if the puck collided and what it collided with, etc.

Now, in order to render things as synchronized as possible both clients must communicate with the server as frequently as possible, because the server is the source of truth, but at the same time there may be limitations that can cause you to not  be able to receive/transmit in each frame, however the game must maintain its fluidity, so your client must be able to decide what to do when it does not receive information from the server, and then communicate to the sever what happened so it can re-sync stuff.

What happens if you get disconnected? the client should be able to determine if it lost it’s connection with the server, and have some way to get you out of the game in a way that won’t make your game crash. You might get substituted by a bot, so the other player can finish, but that is up to the server.

Now let’s complicate things a bit, imagine that somebody invented a extreme version of air hockey combined with pinball “Pinbockey!”, there might be obstacles that react to the puck when they get hit, furthermore, imagine that the placement of these things can be customized, and some players create special tables.

When this happens we will have another complication, you won’t have all possible maps in your game, the server must transmit those maps before the match starts, and your game should be able to read that information and render the map as envisioned by the level designer in reasonable time. Asides from just rendering the table, it must be able to understand what events could happen, and how do those events look and sound, all of that  “protocol” should be established and the client must understand how the server will communicate the state to it, so it can render things correctly, play the appropriate sounds, etc. The main job of the client’s program is to display the world as consistently as it can to all the players, so all of them are playing the same game, and at the same time respect the physics and rules of the game in between the moments when it gets updates.

So far we have talked about a game with two players, but MMOs by definition have many players, and they could be doing things like jumping, running, engaging in conversation, interacting with the world, etc. Some of those things should be visible, and/or audible to you, each running client should create a representation of that reality which immerses each player in a single cohesive world.

Spycursion is a game with some particularities, you can hack, which means you will have devices that will connect to other devices, and that means that you will have GUIs that should react to devices, some nearby and some remote, which may at the same time be visible to other players,the client in many cases will just be a dumb terminal, or GUI and all of the things happening in this interface actually are communicated to and from the server. All of that must happen while keeping the physical world around you going, because Spycursion is also an espionage game, and people around you will be watching and hearing you and in some cases trying to gain access to your devices.

By this point you might be able to get some idea of what happens in the server and what happens in the client, the only thing I have not mentioned is that any program that is capable of showing a representation of the world that is understandable by the player could be a client for that game, so potentially several clients could be developed for a single game, and the visual appeal and performance of the client has a great deal of influence on the success of it.

I hope this was interesting to you, I tried to keep it non technical so all interested parties can get something from this post, we might revisit some of these ideas in future posts.

I hope you have fun and learn something everyday,  and keep an eye on this project, which will give you an opportunity to do both,



Posted on

Viruses, Errors, and You

Not that I would ever wish this upon anyone, but let’s pretend that you just became one of the unlucky 1.5 million people (at the time of this writing) to be infected with COVID-19.  Since this post is primarily about error handling — don’t worry, I’ll get there — let’s think about the errors that may have occurred up to the point when you became sick. In other words, what went wrong, what mistakes were made, to cause this undesirable end result.

    • Maybe Patient Zero was a jerk who coughed on dozens of other people, accelerating the initial spread.
    • Maybe China’s government bungled the initial response, allowing the virus to leave its borders and/or misleadingly downplaying the threat.
    • Maybe your own government didn’t act quickly enough to enact social distancing and ramp up testing.
    • Maybe testing was ramped up, but there weren’t enough supplies or staff members to process those tests.
    • Maybe one of your coworkers or family members ignored social distancing rules and got you infected.
    • Or maybe none of the above happened, and you just screwed it up for yourself by trying to break the doorknob licking world record.

Naturally, at the top of that list, we could put “COVID-19 came into existence.” And with the exception of that mutation “error,” what all of the above problems have in common is that they are preventable. If the right people take the right actions, at the right times, the number of cases of COVID-19 — perhaps including yours — can be reduced.

By this point, however, containment isn’t practical, and with the long incubation period, it may never have been. (You could argue that, if China had acted aggressively and early enough, COVID-19 could’ve been completely contained. My analogy kind of breaks down on that issue, so I won’t get into it.) In any system that is large or complex enough, you will inevitably have failures — people make mistakes, governments dawdle, our scientific understanding proves insufficient. Members of society at large will get sick, with or without showing any symptoms, and everyone else has to figure out how to deal with that.

What does all of this have to do with Spycursion? One of my (many) quarantine projects has been improving the game’s error handling. Just like with a pandemic, any large software project has a massive number of moving parts, any number of which can and will fail. Preventing failures entirely is a lost cause, just like relying on 7.2 billion humans to follow hygiene guidelines flawlessly is a lost cause.

To hammer this point home, consider a few possible sources of errors we, as the game developers, might have to deal with:

    • Server-side infrastructure problems, such as a database becoming unavailable
    • A game client making invalid API calls
    • A game client making valid API calls that violate game rules — in other words, a player trying to cheat
    • Some poorly-written Slang software making an incorrect function call
    • A player running a piece of Slang software incorrectly, like putting “1 / 0” into a calculator program

You might notice that some of these error sources are entirely out of our control! We cannot control what players will do, but we can make sure that Spycursion still does The Right Thing™ when faced with an unexpected situation. This is the essence of error handling.

But what exactly is The Right Thing™ when it comes to, say, a database failure? What happens when the game server calls login_player and the result is error: database unreachable? We don’t just immediately show this message to the player verbatim, partly because it’s meaningless to them, but also because the problem may still be fixable. Ideally, this database will have a backup copy ready to go, so the code can try the same call to a different database server and everything is hunky dory. If the backup is down too, then the player can’t log in, and we have no choice but to tell them something. So that message error: database unreachable bubbles up through a series of constructs, which I won’t get into details about but often looks something like try { stuff } catch(errors) { other-stuff }. (Common Lisp has its own powerful tools for handling errors, but that’s another post for another time.) In the end, the player receives a less confusing message, like: “Oh crap, something went wrong with Spycursion, but don’t worry, we’re gonna fix it ASAP, promise, please don’t talk smack about us on Twitter, kthxbai.”

The Slang calculator program is an interesting scenario, and somewhat unique to Spycursion, because it deals with player-written code. We do provide (simple) features in Slang to deal with errors and warnings, so a well-written calculator program would throw a “divide by zero” error on its own. But there’s always the possibility of a poorly-written calculator that forces us to deal with that division by zero. In that scenario, we still handle the error, but not necessarily without a penalty to the player who wrote the faulty Slang code. (Don’t say I didn’t warn you…)

In all cases, a lower-level error bubbles up to higher levels until it is either handled or we give up and send a pleading message to the player. Going back to our unfortunate COVID-19 scenario, perhaps you can see a similar process playing out:

Error: Virus created

Error: Containment failure

Error: Not enough tests

Error: Unable to process tests

Error: Social distancing breach detected

Error: Too many doorknobs licked

Just like a well-designed software system, a well-designed public health system will reduce illness by handling these “errors.” Well, except that last one. There’s still no cure for stupidity.



Posted on

Realistic Hacking

We have talked about realistic hacking in Spycursion a lot of times, but what exactly is hacking and what do we mean by realistic? Let’s answer those questions one at a time.

Hacking

Frequently when people hear about hackers they imagine a shady figure with a hood trying to steal your bank account information or get into your e-mail, but it is more about applied knowledge and outsmarting systems to get what you want, some times maliciously (black hat hacking) and some times legally (white hat hacking). In general when people think about hackers they are thinking of coders that can cast magical “spells” which allow them to do almost anything, changing traffic lights, changing their college grades, moving money around, or even get heart attacks to people using connected pacemakers.

In modern days a lot of these things can be realistic, if the systems can be controlled remotely chances are somebody can design an attack that will allow them govern the system, and make it do things, some of which are unexpected even to those who designed the system. But hacking is not limited to that, however in a game we need to limit things to what can be achieved inside our simulation of the world.

Realistic Hacking in Spycursion

To disrupt or infiltrate a system we need to know how things work, how do clients access the system, what happens when a system receives a request (or a lot of them), how do systems protect against over-lookers, and how can you overlook the information coming and going from the parts involved in a conversation.

Once we know how things work, we have to be ingenious to know how we can make them do what we want or stop doing what they are supposed to do. So in order to have realistic hacking in our game, we need to have systems that are accessed in a similar way than real systems do, with existing protocols, and with limitations similar to those of real systems.

Where do systems, break?, the answer could be in the physical layer, in the low level communications, in the protocols, in the limitations of the server (an its underlying OS), and a lot of times in the people using the system. So a player in our game should be able to create attacks to any of those layers. This means that:

There must be servers, clients and peers interconnected trough an internet, with messages going to a from servers, some times through wires, and sometimes wirelessly, IR, Wi-Fi, bluetooth, sound, lasers, etc. Operated by real people and/or NPCs.

The Players/NPCs should have an environment which they live, and where they have personal possessions, garbage, social interactions, etc. In the world of hacking information is crucial, and it is not always obtained electronically. Some times you have to spy, eavesdrop, invade private property and look for clues, go trough somebody’s garbage to gain information, go to social media to gain insights about a person, impersonate trusted entities in real life or electronically, etc.

So, the Spycursion world should allow you to do physical hacking, tap lines, connect usb devices to servers or computers, set up wireless hot spots, intercept radio frequency, infra red, or other types of transmission media. It should also allow you to use public connections and code to intercept communications not intended for you and be able to use that information.

Each server should have speed limitations, and be limited in the number of connections it can keep alive, and should obey some rules on what to do with incomplete communications, disconnected clients, and other disruptions.

Finally, our citizens (some of them NPCs) should have the same type of vulnerabilities as real persons, use obvious password, leave unprotected ports opened, be vulnerable to phishing and social engineering. and make silly mistakes as leaving the computer unlocked and unattended, or send passwords in chats or phone conversations, etc.

An example, Man in the middle attack.

Real Life

There are several ways this can happen in real life, we will talk about one. When you connect to a hotspot on an airport or restaurant. You normally open turn on your Wi-Fi and start looking for hot spots and look for something that looks right, “theBurger-guests’ for example, what you are in effect doing is connecting to the Wi-Fi router which in turn connects to the internet. Now, most phones can be turned into a hotspot to allow you to use your phone carrier’s internet to connect to the internet from other devices (normally your laptop), it also allows you to change the predefined name so you could name it “spacecowboy” or whatever you want, and you can set up the security for it. So if I were to go to theBurger and rename my hotspot “theBurger-guests’ and make it insecure, anyone could connect to it. and in fact many people would, because they trust it is a Wi-Fi for the customers. Now you have all the internet traffic from the people who connected going through your phone, and if you have the right apps you could redirect them to wherever you want, or sniff their packets (since all the traffic is going through your phone you can see what is being sent and received for all the users, that is what we call sniffing). You can imagine you could do all sorts of malicious stuff then, like serving a custom page of your own when somebody asks for amazon.com, and try to trick them into giving you personal information under false pretenses, or try to add your own items into their shopping cart and change the delivery address, etc.

Spycursion

Now, in game if you have a device that can serve as a Wi-Fi hotspot, that could be simulated by a process that has a certain number of channels that can receive and send information, the game will know if a player or NPC is sufficiently close to it to discover it and/or connect to it. Imagine you want to spy on employees of a data center, and they normally go to the same coffee shop ‘AstroDollars’, then you could set up your device somewhere in the shop and have all packets dumped into a file that you can later analyze with your computer. You could write the “sniffer” and “analysis software” yourself, or get it in an in game store, either way is going to be “Slang”. Now since Spycursion is as much about defending as it is about attacking, then, knowing what you know, you would not be connecting to any random hotspot you find, but you also have a need to connect to the internet on the go, so you have to figure some countermeasures, you could for example use your laptop to send and received encrypted messages to your home computer, and have your computer be the one connected to the web site you want to reach, effectively acting as a tunnel, then anyone looking at those packets should be able to un-encrypt the traffic in order to see any info in it. Again, in game your computers are processes that can receive and send messages, in this case with a program that allows them to receive and send encrypted messages between them, these programs could have different types of encryption with different strengths, and demand on resources.



Posted on

Designing a Video Game API

We’ve talked a bit about Spycursion and its open source model, but we haven’t really delved into how that will work from a practical standpoint. In this post I’ll talk about application programming interfaces, or API‘s, with the goal of helping you understand some of the decisions that go into designing an API for a multiplayer game like Spycursion. Please note, this information is intended for a more general audience, so many of the details have been simplified. This post is an introduction, not a reference manual!

What is an API?

An API is a set of rules and agreements, through which developers of various software components agree how those components will communicate. You can search the internet for API documents of all sorts to get a feel for what typically goes into those documents, but if you’ve never tried writing code for an API before (or writing code at all), you may get lost in the jargon.

Instead, let’s look at a simple real-world example…

Server: Hello, and welcome to Foodz R Us! Can I get you anything to drink?
You: Purple cow.
Server: Excuse me?
You: Purple cow.
Server: Err, I’m… sorry, we don’t have that on the menu. Would you like-
You: PURPLE COW!

When a server at a restaurant asks if you’d like anything to drink, they are usually expecting you to name something on the drink menu, or perhaps to ask a question. Here, though, you violated their expectations and the server’s “API” returned an “error.” (I personally have never seen “purple cow” on any drink menu, but if you do, let me know!) In computer-speak, this example could be converted to something like…

Server:

{ "request": "drink_choice" }

You:

{ "drink_choice": "purple_cow" }

Server:

{
  "error": "purple_cow not found",
  "drinks": [ "water", "tea", "milk", "juice", "beer" ]
}

{ "request": "drink_choice" }

You:

{ "drink_choice": "purple_cow" }

Server:

{
  "error": "purple_cow not found",
  "drinks": [ "water", "tea", "milk", "juice", "beer" ]
}

{ "request": "drink_choice" }

You:

{ "drink_choice": "purple_cow" }

Seasoned programmers will recognize that the messages above are in the popular JSON format. Certainly not all API’s use JSON (Spycursion’s does), but I will be using it throughout this post, for its simplicity and readability.

Movement

Just about every multiplayer video game will, at some point, involve you moving one or more characters from point A to point B on a game map. To keep things simple, let’s say our game map is a 2D grid made up of integer coordinates. (You know, the grids your high school Geometry teacher made you draw.) When you first start the game, you don’t necessarily know your character’s starting point, but the server should have that information, and so it lets your game client know:

{ "player_position": [ "x": 5, "y": -2 ] }

Now that the client and server are synced, you can move your character around the map. In a flat, boring world with no obstacles whatsoever, your client could just do something like this:

{ "move": [ "x": 3, "y": 1 ] }

But where’s the fun in that? In Spycursion, as in most other games, pathfinding may be required. Walking directly from A to B in a straight line might have you plowing through buildings or getting run over by a car! So the client, or the server, or both, will need to be able to navigate your character around obstacles. In practice, you would likely rely on the client for pathfinding, then convert that into API requests. If you as a player click the point (3,1), but there is an object in the way, the actual API requests might look like this:

{ "move": [ "x": 6, "y": -2 ] }
{ "move": [ "x": 6, "y": 1 ] }
{ "move": [ "x": 3, "y": 1 ] }

But what happens if a player hacks their client to just ignore those obstacles? What does the server do with a request that seems to defy the laws of physics? This is where collision detection comes into play, and any well-designed game server will include it. Basically, upon a player trying to run through a tree, the server sends a message to said player’s client…

{ "player_position": [ "x": 5, "y": -2 ] }

… as if to say, “Sorry, that thing is impermeable. You’re still at (5,-2).”

REST

No.

… Sorry, I should explain. REST stands for representational state transfer and is very often used in a sentence with “API.” Without getting into too many details, REST is an architectural style that is common among web API’s, and one of its more notable features is that the server does not store any client context between requests. In other words, every client request has all of the information that the server needs in order to process it. In a RESTful API, movement requests would look something like this:

{
  "from": [ "x": 5, "y": -2 ],
  "to": [ "x": 6, "y": -2 ]
}
{
  "from": [ "x": 6, "y": -2 ],
  "to": [ "x": 6, "y": 1 ]
}
{
  "from": [ "x": 6, "y": 1 ],
  "to": [ "x": 3, "y": 1 ]
}

A little verbose, but not terrible, right? Well, think about that little cheater with the hacked game client:

{
  "from": [ "x": 5, "y": -2 ],
  "to": [ "x": 3, "y": 1 ]
}
{
  "from": [ "x": 6, "y": -2 ],
  "to": [ "x": 4, "y": 7 ]
}
{
  "from": [ "x": 666, "y": 666 ],
  "to": [ "x": 666, "y": 666 ]
}

A stateless API would give the kid free reign to teleport around the game map! Naturally, this makes no sense for Spycursion, and you should treat any multiplayer game advertising a “REST API” with extreme skepticism.

Vision

I don’t believe we’ve talked about this, but here’s a new Spycursion tidbit for you — every player and NPC in the game has a vision cone. You can only see characters about 135 degrees in front of you (buildings and other stationary objects are always visible), so it matters which direction you’re facing. If you sneak up behind another player, they can’t see you without turning around. The API request to change where you’re facing is straightforward:

{ "look": 1.570795 }

You might recognize that number — yes, here are those Geometry nightmares again — as half of pi, telling the server that your character is now looking at pi/2 radians, or 180 degrees, which in this case just means “due south.” When you turn around, you might discover another player trying to sneak up on you, about which the server now sends a warning:

{
  "player_id": 66666,
  "player_position": [ "x": 5, "y": -3" ],
  "player_looking": 0
}

In reality, you wouldn’t be sent the opposing player’s actual ID (see below), but instead a lot of information about their appearance, and perhaps an indication that you’d encountered them before. Again, we’re keeping things simple here.

Interaction

Another MMO trope: Interacting with other characters. (I know, it’s awful.) In most games, if you wanted to converse with and/or hit on another character, your client might send a request like this:

{
  "action": "talk",
  "player_id": 66666,
  "message": "Hey, stop sneaking up on me."
}

And that’s all well and good, if your client has the player’s ID, which it probably does, as it was probably already sent by the server. Spycursion, though, is a bit different. We can’t tell you a given player’s actual ID or username, because of the possibility of disguises. (This is a spy game, after all!) If you knew that little Johnny’s player ID was 66666, and this fact was always shared by the server, then little Johnny could never truly fool you. Instead, we have to assign a temporary ID for every character who enters your vision cone. This is shared between client and server, until enough time has passed, or you leave the map, at which point you “forget what they looked like.”

If another player starts a conversation with you, your client gets a message like this:

{
  "player_id": 1234,
  "says": "I heard there was a new Spycursion blog post today!"
}

Other spy-like requests look similar, except of course the server doesn’t notify your client that someone has just planted a bug on you. 😉

Hacking

This post wouldn’t be complete without me talking about the API details behind what happens on your character’s computer screen. That said… there actually isn’t much to talk about! Spycursion’s in-game internet will offer many services which can essentially be divided into two types — “hackable” and “unhackable.” The “unhackable” services will have their own API specifications, while the “hackable” services will consist of in-game programs written in Slang. All of the Slang programs follow one simple API request:

{
  "command": "rm",
  "args": [ "-rf", "/" ],
  "target": "0123:4567:89ab:cdef:0123:4567:89ab:cdef",
  "session_key": "0123456789abcdef"
}

The target and session key are just for verification that you are logged into a remote host, and for allowing you to have multiple terminals running at once. That request includes every (non-client-specific) command you’ll run from the terminal, including network tools, the Slang compiler, and your programs themselves.

We haven’t fully decided which services will fall under which categories. Transparently, we would love every service to fall under the “hackable” category, but it’s just a question of feasibility and what we actually want players mucking around with.

 

Spycursion’s API documents aren’t yet ready for public release (and even if they were, the server isn’t ready for public release, so you wouldn’t be able to do much with them). In the meantime, hopefully this post has taught you a few things about video game API’s, and maybe even motivated you to design your own!



Posted on

Anomaly Report #73 (CASSI 001)

From: Burdinjaun
To:                    

Subject: [TS//SCI] United States Armed Forces, Cyber Directorate — Possible Security Breach
Status: Investigation in Progress
Interviewer:                         , Military Investigator

Verifying… [100%]

Deciphering… [100%]

 

 

Interviewer: Please verbally express your consent to record this interview.

Tomás Lednura, PhD: I consent.

Interviewer: Can you summarize the facts of what happened on Oct. 24th, in the Annex B Conference Room 109, at 13:34?

Lednura: OK, as I recall it… Hmm, 17 people were summoned to the room due to an urgent security breach. 13 were there when the meeting started and the doors were closed. Four were high ranking officers, six were cybersecurity experts. The rest were CASSI developers.

Interviewer: For the record, CASSI is the Computer-Assisted Security Surveillance Intelligence. Do you recall all the names?

Lednura: You should have them in the written report I sent… (trails off) Jack from cyber security, my boss, called the meeting. I made the discovery of the information leak, and he restated what I had told him. At 13:01, one of the servers hosting CASSI started extracting data from the other servers and sending it to external sources. There were no known intrusions at the time, as verified by extensive malware checks. The data transfer processes were not spawned from a cron job or any other server configurations, and resisted all attempts to kill them. After getting everybody up to date on the situation, an engineer from the AI development team gave an assessment… (pause) Lisa, I think. Anyhow, she said that CASSI was only capable of monitoring communications and looking for possible national security threats. Any alarms would be relayed to a GUI in a room where some analyst would continue the work, and the system had no volition or any I/O capable of moving files or sending messages to the outside. I guess she could give you a better summary.

Interviewer: Sadly, we can’t ask her, nor anyone else. After the events of that day, one by one everyone in that room has died or disappeared. Except you, Dr. Lednura.

Lednura: What? How is that possible? I talked to Glenn a week ago!

Interviewer: Please stay on topic. We’ll talk about Mr. Lewis’ disappearance later. We need all the facts first.

Lednura: (sighs) I think those are all the facts any of us had prior to the meeting. During the meeting, we found that large amounts of data were being transferred. There was no rhyme or reason as far as security classification, but most of it was generated by CASSI processes. Like I said, every time we killed one of those processes a new one replaced it, and no other countermeasures worked, either. Gen. Miller asked for a summary; we told him what we knew and showed some evidence illustrating what was happening. That’s when I suggested taking the server offline, but there was a lot of debate about that. The CASSI guys were adamant that there was no way it could be responsible for the situation.

Interviewer: Was it?

Lednura: The evidence was inconclusive, but it was strong enough to suggest that turning off the server could stop the leak. I explained that, and Gen. Miller gave the OK to shut it down, even with the devs protesting about its state being corrupted, losing generations and all of that. As soon as he gave that order, the alarm went off, and the monitors showed the activation of something called Protocol Z14. I didn’t even know what Protocol Z14 was, I still don’t. But I distinctly remember hearing the big doors locking.

Interviewer: The big doors?

Lednura: That’s internal jargon for a complete lockdown, basically. Gen. Miller gave us this confused look and said that Protocol Z14 could not be activated without his OK. But we were all locked out of our systems.

Interviewer: Aside from that, did he reveal anything about the nature of the Z14 protocol?

Lednura: No, we’re all trained to avoid asking too many questions. But a voice recording sounding with the alarm said we had five minutes before the whole floor was incinerated, so we were frantically trying to override the security systems preventing us from getting out, or at least the ignition systems about to burn us alive. (sigh) But it was useless. After a few minutes we gave up and accepted our fate.

Interviewer: Just like that?

Lednura: What else could we do? We were in a concrete basement, with two-meter reinforced walls, a single exit sealed by a solid steel door, no working overrides, and a (expletive) incendiary bomb about to go off!

Interviewer: OK, and-

Lednura: Gen. Miller was starting to say something when the alarms ceased, and a green notification on the monitors signaled that Protocol Z14 had been suspended. Everybody was cheering, but the general just had that same confused or worried look on his face.

Interviewer: I can tell you this much: Protocol Z14 was designed to not be stopped, paused or suspended in any way. Once initiated, there is no way to stop it, by design.

Lednura: I guess that’s why he asked right afterward who had stopped it. We were all as puzzled as he was. Then a bunch of code appeared on my screen-

Interviewer: That’s enough, thank you. Now let’s talk about Mr. Lewis’ disappearance.

 

(click sound; end of recording)



Posted on

Spycursion and the News

I’d originally planned to pick a relevant news story and tie it to Spycursion somehow — “Hey look, you can steal Bitcoin too, here’s how you might do that,” etc., etc. But it turns out there were so many news stories that I couldn’t just choose one; that’s also why this month’s post is late. (That’s my excuse, and I’m stickin’ to it.) The plethora of cyberespionage-related news stories inspired another idea, which I’ll get to in a minute. As for the Spycursion update… I decided to go meta.

If you live in the United States, you’re probably aware of the allegations by the intelligence community against Russia for attempting to interfere in our 2016 (and 2018) elections. To the best of our knowledge, Russia didn’t do this by hacking voting machines — instead, they hacked our minds. They spread disinformation targeting both ends of the political spectrum in an attempt to divide Americans and sow chaos.

Consider where you get your news. If you’re like most people, you probably get a significant chunk of your news online, and a significant chunk of that from social media; the other largest news source, especially for older people, is TV.

Now, if you’re a disinformation specialist and you want to spread some salacious-but-obviously-false rumor, where do you go? The New York Times has over 130 million monthly readers, but that would be difficult to get into… pesky fact-checkers and all that. No, if your story is actually fake, no reputable news source would publish it (we hope) — you have to get in through a backdoor. One of those backdoors is social media. So you, being the sleazy fake news artist that you are, make a troll post on 4chan. Some legit-sounding but conveniently anonymous users “verify” it, and then some Twitter bots pick it up. From there, it spreads, with trolls fanning the flames and useful idiots spreading it around, and you’re off to the races.

There’s another backdoor, though, one that’s arguably even more insidious. This backdoor is mostly effective with stories that are half-true, or true, but with a particular spin on them… but it doesn’t have to be limited to those stories. The NYT alone has about 1300 employees. Let’s say you identify a shy young election reporter who made the rookie mistake of saving sexytime videos on her old, unpatched iPhone. Insta-blackmail! Now, her editors might catch anything too egregious you try to publish, but go up a couple of steps in the chain of command, and you could virtually guarantee the publishing of any semi-legitimate story that you want. And that’s not all! News organizations don’t just stick with their own stories, of course; they source from others. What gets published in the NYT also goes to Reuters, CNN, the gossipy lady down the street, etc. Congratulations! You’ve effectively blackmailed the entire media industry.

I don’t believe we’ve discussed it much, but we’ve designed a media system into Spycursion. It will be basic at first, like publishing stories about major hacks that players do, and other random events. The ability to blackmail journalists may come later. In order for that to be useful, you’d first need a good reason to do it — elections, stock market, laws, etc. All more advanced and ambitious features… but all incredibly awesome. 😉 You may even have the ability to play as a journalist, working for a news organization, deciding what stories to publish or not to publish… and being targeted by disinformation specialists.

Spycursion News Network

Now, for that other inspired idea: The Spycursion News Network! We will be sharing out real-world news stories about cybersecurity, espionage, and related topics, through this Twitter account. (The way things are going… we’ll probably post multiple times every day.) And yes, as the game and the media system progress, we’ll share some fake stories from within the world of Spycursion, too. For now, it’s all real, though. If you’re interested in these topics, give that account a follow!



Posted on

Project Update for June 2019

Common Lisp: How deep does the rabbit hole go?

Being that both Scott and I had to go back to getting jobs to put bread in the table things are going slower than when we were both 100% dedicated to the project. In the last update Scott mentioned some ideas we had around re-structuring the client, as you may imagine we can manage to keep the server side purely in Common Lisp, but the client side is a bit different, we need to create UI elements, and 3D graphics, as well as adding assets. We talked about moving away from Qt, not because Qtools are not capable, but mainly because in it’s current state UI elements cannot be rendered in the OpenGL window, which creates a disjointed experience for the user.

So we started an internal discussion to evaluate the future of Spycursion’s development, things that we wanted, things that we needed contrasted to what we have and what we need to implement, and how home grown that implementation needs to be, of course we want to use existing solutions where it makes sense, but we have our own philosophy around development.

Quick story, we considered going down the path of using Godot, using gdnative to be able to program all of the game logic in Common Lisp while using the game engine capabilities to import assets, create the world, apply physics, etc. I even started the task of creating the bindings1 using CFFI. All of this seemed like a good contribution to the lisp game developers community, and there are plenty of Godot tutorials on line. So asides from having to redo everything in Godot, what stopped us?, it is not Lisp far enough down.

If we make it so we can program Godot’s game logic in Common Lisp we are only layering a coat of usability on top of an otherwise C++ stack. Lisp Hackers could only contribute to the engine through plug ins or functions that work with the API, but if they want to go deeper they have to switch languages. So the fair question now is, how far down we can go in our current stack before we have to switch languages?, after all a lot of what we do in game development is done through FFI. The answer is, arguably, farther down than if we just wrap Godot, of course with some trade-offs. So what will Godot do for us, here is a non comprehensive list:

  • Asset management, sounds, music, sprites/models.
  • A visual environment to create levels and assign properties to in-game objects (walls, ground, etc).
  • A physics engine.
  • Custom Shaders (with their own language).
  • GUI which can be layered on top of the game.
  • A simple scripting language for the logic.

What we have now with our prototype client:

  • Asset management, mostly 3D models only, character animations are rendered correctly, 2D sprites can be imported as planes with texture.
  • A non visual environment to create complex scenes.
  • Custom Shaders using the excellent CEPL library created by Chris Bagley (akaBaggers), shaders are written in Vari which is very Lispy.
  • The complete Common Lisp language for the logic.

The Missig parts

Asset management

We have not added any sound to our game yet, so we haven’t dealt with importing and managing sound including music, however there are several options out there like Pavel Korolev’s (aka Borodust) bodge-openal, or Nicholas Haffner’s (aka Shinmera) harmony, and some others. So we have native options we can integrate in our client with relative ease.

Scene management

Although I listed the “non visual environment” I guess I should mention two things. First, in our current work flow I was able to create a scene in Blender directly, and then import the elements one by one in the SC client replicating the placement from blender automatically. So in a way we have a visual environment to create the scene. I think however, that using CEPL it should be simple enough to create a solution where you move and rotate things around as needed. So far we haven’t got a need for that.

UI

We are working currently working on integrating bodge-ui (suggested by our good friend Borodust), which will allow us to layer UI elements on top of the game, this is what will substitute Qtools, I’ve made some basic testing and the UI renders well in Linux and Windows, I can’t check Mac but we will get around to testing more thoroughly once we integrate things in the Spycursion client. We also talked about having the users be able to create “web pages”, the solution in Godot seems to be using a combination of bbcode and the GUI, which got us thinking, we don’t really need a full blown HTML render in game, in fact is safer to use something like bbcode but with the addition of forms.

Is there need for a Common-Lisp game engine?

We have been constructing a game, not a game engine, using loose parts created by great programmers, by programming the code to be modular and reusable we get a similar effect to having a full blown engine, as an example. if we need to add another scene we just create a scene file, place the assets in a convenient place, and declare things and we have a scene ready to be rendered. I talked about Shinmera’s harmony, and Borodust’s bodge-ui, these wonderful programmers are already giving us the building blocks, so you can make your own mix and match, or you can choose to use their full solutions i.e. Radiance, CL-Bodge, etc. We even thought about going 2D with Dave O’Toole’s (aka dto) Xelf.

Ongoing work

There is a lot going, very slowly but going, we are refactoring the client to make the code modular an easy to reuse, some manual steps are being moved to CLOS methods (before, after and around), some hard coded values are being moved to special variables, etc. The code has to be easy to maintain, and our initial approach needed some refining. I’ve been working mainly on the client, and Scott more on the server, and a lot of what we have been doing is “invisible”, but we are doing what must be done. Hopefully as we add/refine the visual aspects of the game we will be able show the progress instead of just telling you about it.

Patreon

We want to publicly say thanks to our Patreon supporters, although we cannot live from their contribution some of the game expenses will be alleviated, and it shows us that there is still interest on having this game become a reality.

Footnotes

1. Borodust already did Godot’s bindings



Posted on

Spycursion Update (May 1, 2019)

It’s been about a month since our Kickstarter campaign ended and we launched our Patreon page. As people who have watched exciting projects rise and fall — often without even saying goodbye — the last thing we want is to leave Spycursion dormant for months while our fans wonder what happened to it and if we will ever return. (If Spycursion ever does fall into the bit bucket of history, it won’t be because we ghosted you — it will be because we went out in a blaze of glory.) To that end, our goal is to share a progress update roughly once every month, even in months where not a whole lot has happened…

… like this one! As I hinted in the last post, both Mauricio and I have had to “take some time off” to focus on useless trivialities like, say, paying the electric bill. Aside from that, we’ve been zooming out a bit to discuss the Spycursion architecture and how to tackle a development roadmap in our new post-Kickstarter world.

Ooh, Shiny!

Can I be honest? You know all those parts of the game that you’ve probably seen from our trailer? Those are my least favorite parts. The parts of Spycursion that I enjoy talking about and sharing the most are, well… the parts you can’t see. There are good reasons for that. Spycursion was never meant to be a graphics-first game, and in fact, I hadn’t even planned on making it 3D! My original vision of Spycursion was as a 2D isometric world. It evolved into 3D throughout the process of development and commissioning assets, as we decided that 3D would allow for greater flexibility in terms of feature additions and look-and-feel modifications.

But really, we should go back even further. In July 2018, we published this post containing the very first public Spycursion screenshots (with the exception of this very, very early device UI screenshot, in Nov. 2017). At that point, the game had been in development for nearly a year. We hadn’t shared screenshots because we weren’t focused on anything that could be screenshotted. Then, we read pieces from some wise(?) internet denizens who told us that, in order to have a successful Kickstarter campaign, we would have to make shiny things and show them off and generally become glued to social media. (Anyone who’s ever worked on an indie game knows where this is going.)

Hence, our focus shifted from server to client, and more or less remained there until the Kickstarter launch. I don’t know whether this was ultimately the right decision, but I do know that not focusing on the client certainly wouldn’t have helped that campaign. In any event, I personally feel that the visuals in Spycursion’s trailer turned out well, but not as well as they could have. There was a degree of crunch in the weeks and months leading up to February, and as a result I didn’t feel we were able to give the client the polish it deserves. That’s why one of our first post-Kickstarter discussions was about…

(Shiny) Client Architecture

If you’ve followed our blog, you know that the Spycursion client relies heavily on a couple of Common Lisp frameworks — CEPL and QTools. What you might not realize is that, within our code base, these two frameworks never even touch each other. It’s almost as if we have two separate clients that just happen to have the same name. That makes some sense, of course, because they have very different purposes. But let’s say that we want to display a device UI (a Qt QMDIArea, if you’re fancy) partially overlaid on top of that 3D world, rather than in a completely separate window. I’m not sure there’s a good solution to that. (QtOpenGL comes the closest, but we dismissed that because it would be difficult to use with CEPL or other GL frameworks, plus it had some problems on OS X.) Qt also adds some bloat that we’d rather not have.

Though we’ve discussed it before, without the time pressure of Kickstarter, we’ve decided to begin replacing QTools now. And that replacement will be… Nuklear! Why Nuklear? Well, for starters, look at these two screenshots and tell me which one fits better in a game about cyberespionage. (Yeah, I thought so.) In seriousness, what’s attractive about Nuklear is that it’s simple and embeddable, and doesn’t try to take over your entire application.

Now, the astute and tech-savvy reader may be thinking, “Wait, Nuklear is written in C, not Lisp… does it have any Lisp bindings?” Well, yes, yes it does! We’ve decided to give cl-bodge a shot, specifically the UI module, and see if we can make a workable device UI out of it. Another potential benefit here is that we could get away from traditional desktop UI paradigms of the sort enforced by Qt, and design something more unique. It’s too early to know what that might look like, but I’m excited about the possibility.

If cl-bodge works well for us on the UI, we may also use other modules of it for other purposes. It is possible that we will eventually replace CEPL, but I think the more likely outcome is that we blend the two, taking advantage of cl-bodge where it makes sense and working with CEPL as a friendlier alternative to raw OpenGL (cl-opengl).

More Shiny?

Between paying the bills and replacing QTools (not to mention me having a strong desire to hide back in my server code), it may be awhile before we can share more screenshots of any quality. It will definitely be awhile before we share more videos. When we do, we’ll try to include some upgrades to the 3D world, as well. We picked up a fair amount of technical debt during pre-Kickstarter crunch time, so performance improvements are definitely in order. With performance improvements, we can add more “bling bling” (to use the words of an influencer who didn’t want to share Spycursion because it didn’t have enough of that). And if we get enough funds through Patreon, we could add new 3D art as well.

What Kind of Shiny Would You Like to See?

As always, we invite our Patrons and followers to help shape not only Spycursion itself, but how we share it with you. What do you think about the technical changes outlined above? What do you want to read about in future updates? What would you like to see on our Patreon page?

Let us know!



Posted on

Out With the Old, In With the New

The Launch

On Feb. 26, we launched our Kickstarter campaign in the hopes of funding the rest of Spycursion’s development. Our spirits were high (even if our energy levels weren’t), we hit 2% funding within a few hours, but most of all, we were just excited for our studio to move into a new phase of its life. We had lined up PR contacts and were running a really clever — perhaps too clever — marketing campaign. At that point, all we had to do is wait for the inevitable surge in backers.

Early Bird

Admittedly, we should have seen the signs. By Day 3 I was beginning to, but to be completely honest, I spent most of the first three days of our campaign so exhausted from the preparation of it that I could do little else but refresh social media feeds and ask Mauricio why more backers weren’t coming in. The press weren’t responding to us in nearly enough volume, our ad campaign was a total flop, and our mailing list on its own wasn’t large enough to drive the early traffic we needed. We had focused all of our energy on that 72-hour period, and it netted us a whopping 4% funding. I was about to call a pow-wow with Mauricio and Dan to figure out how we could save this floundering campaign. Until…

CRACK!

Did you hear that? That was the sound of my wife breaking her wrist (in a very complicated and interesting fashion, she says). So, yes, I spent much of the next week focusing on her rather than our Kickstarter campaign.

The Doldrums

By the time I got back into things, our campaign hadn’t advanced much further and we all agreed that it was more or less doomed. That’s when we made the decision to move to Patreon and shared this update on the Kickstarter page. Since that update, Mauricio and I have been working on said Patreon page, discussing our new development plans… and looking for jobs. C’est la vie.

The Future!

As you may have already gathered, development on Spycursion will continue! It will be at a slower pace, no doubt. We almost certainly won’t be able to hit the Q4 2020 goal that were aiming for with the Kickstarter campaign. On the upside, if we can get our sustainable funding situation worked out, then you’ll have happier, less stressed-out devs working on the game — which means greater productivity and fewer bugs!

For now, we’ve launched a Patreon page to serve as both a funding boost and a barometer of how our fans are feeling about the game. Right now, our membership tiers are very simple: For every month that you back us, for an amount of your choice (more on that in a minute), you get one month of Spycursion gameplay after it’s released. We will certainly add more tiers, and possibly raise the minimum price tier, as time goes on. For now, we’d just like to know what our supporters want in terms of rewards. Let us know if you have reward ideas or other thoughts on our Patreon page!

In diagnosing the non-success of our Kickstarter campaign, one of the conclusions we reached is that the pricing and value we offered didn’t line up. Our pricing was based on a fuller, more polished version of Spycursion that we had in our heads — rather than reflecting the presentation that we were able to showcase at this stage of development. People don’t see what’s in our heads, they only see what we can show them, and what we could show them didn’t justify the price points. (And yet, it clearly did, for those who backed us. A friend pointed out that some backers were willing to pledge over $200 for a game that’s 18+ months away! That’s certainly not nothing!)

This is why the “pay what you want” model we’ve adopted on Patreon is key. We’d like our supporters to think carefully about what they see, what we’ve shared, and our own capabilities, to decide what a month of Spycursion is worth to them… and then tell us! At this point, we’re expecting a lot of $1/month backers, and that’s perfectly fine. Over time, we hope to earn more supporters and more trust. The nice thing about Patreon is that we don’t have to get everything right at the beginning; our fans can help us over a longer time period in molding Spycursion into what they’d like it to be. We hope you will join us on this journey!



Posted on

The basics of game programming in Common Lisp.

Common lisp is a wonderful language, a lot of people tend to think it is a functional programming language, and while that is not a bad thing, Lisp is not a functional programming language, it is a multi-paradigm programming language. While it is not considered good Lisp style, you can program in an imperative way; Why am I telling you all this? well, because people always object that being functional makes it difficult to program games, which is not completely true either, but Common Lisp will work with you and not force you to work around it.

While I am in the theme of fallacies, another false misconception about Lisp is that it does not have libraries, it does have more than a descent amount of native libraries, Common Lisp implementations also have FFIs which has allowed to create bindings to very useful C libraries, and since bindings are commonly not “Lispy” some times we have great adventurers creating wrappers that made them feel like they were written for Common Lisp.

So, What do you need to program a game?, in the simplest terms you need:

  • Output: A place to show text, graphics, lights, sound or whatever your game uses for interaction with the user.
  • Input: Devices to interact with the game (i.e, move the character, move icons, write text, etc).
  • Assets: Different types of things that go into your game (images, models, sounds, maps, music).
  • Constraints: A set of rules that physically or logically constraint the actions that you as a player can do (i.e. physic engines)
  • Logic: A set of rules that affect how the user interacts with elements in the world. (i.e. open a chest, a door, use a computer etc)

    You might argue there are more things, but to me this five make the core of what you need. So lets break this down for Spycursion, which is our game, I will intentionally leave out the socket magic happening between the client and server, which is necessary in any MMO.

    There are two types of interaction in our game, the Computer GUI and Terminals and the 3D world.

    Spycursion Computers

    Input/Output

    The computers in Spycursion are meant to look like real computers, once you start using a computer you will either see a GUI environment or a Terminal, so what we need here is a GUI library, we could (and actually thought about) write one from scratch, but with so many solutions in existence out there we decided to use one with the characteristics we needed, and the winner was Qt, one of it’s redeeming qualities being portability, which is important when you are writing a game that runs in Linux, Mac and Windows. To write our Qt in a lispy way we use the fantastic Qtools library written by Nicholas Hafner (a.k.a Shinmera).

Assets

Qt provides the necessary tools to import icons, images, play sounds, and even render html, so it is relatively easy to get a full screen application that looks like a computer desktop.

Constraints

Constraining a GUI is very easy, you just provide the programs that you want people to be able to use, and what those programs are capable of doing. Constraining a terminal is even easier, you just decide what commands exist and how they interact with others (piping, I/O). In both cases you can restrict user rights, so it ends up being a bit like real life system administration, but with more control.

 Logic

Implementing Logic in Qt is very straight forward, you can decide what happens when you press a button, hit enter, change text, etc.

I won’t provide examples because there are plenty on the Internet.

Spycursion 3D World

Output

Since we are shooting for a 3D world the safest bet is to use OpenGL, Lisp has the cl-opengl library, but that is too verbose, the good news is that Chris Bagley (a.k.a Baggers) created CEPL, which has as a design goal, and I cite “… making the user feel that GPU programming has always been part of the languages standard.”, asides from that promise, Baggers has spend hours doing great videos which he has shared on Youtube.
To run OpenGL you need what is called a surface, which is basically a graphic container, when we started doing this the only stable option for CEPL was SDL2, so we to use cl-sdl2 too. The library, in this setup, is used only to have a display window, but it is capable of a lot more, as an example you can look at sketch, which you could use to create 2D games.

Input

A lot of the things we care about were also concerns to Baggers, and he has a whole ecosystem of related stuff, some of which even come installed with CEPL or are part of it, for Input his contribution is called skitter, which allows us to detect mouse movement, position, clicks, keyboard presses and all that jazz, it can even handle game controllers. While we use it with CEPL it is independent of it.

Assets

So far we have been worried more about showing a 3D world, some menus and text, so I will talk only about those aspects, of course there are options to import and play sound, but I will focus on 3D models. You should be able to use a 3D modeling program to create your assets, save files, and get the data from those files imported into your program, that is the job of the great assimp library, and of course we have classimp which is classier ;-), with that library you can import several 3D model file formats, so you can use a variety of programs to create your model, including free options like Blender, or you can buy them in a format you can use. Of course this only creates a data structure, so you have to do all the wiring yourselves, but you will be able to import models including the animations which you need for your game.

Constraints/Logic

All constrains and logic are programmed in Lisp, with some help from stenciling, this is one of the few instances where we used cl-opengl directly instead of using the CEPL scaffolding. In general Common Lisp is a very potent programming language, so writing logic in it is very straightforward and flexible. We are currently not using physics engines but there are libraries for that too.

Some final thoughts

There are many wonderful tools you can use to make your game, we are just talking about the ones we used, you could use Xelf for 2D games, or Trial for 3D and 2D, there are libraries for text rendering both in 2D and 3D, libraries to do text based games (i.e. cl-ncurses), tools for importing and playing sound. There have been games written in Common Lisp in recent years, and there are Common Lisp game jams (the next one starts on April 18th, 2019), the fact is that writing games has been improving the ecosystem.

If you ask for it, we could expand in some of these themes, but the basics are the same, choose the type of game you want and the look for it, choose the adequate input and output libraries, add suitable libraries for importing assets, use some libraries for collision detection and physics if necessary, add your own logic, mix everything with tons of creativity, and you’ll have a game.