Why Large Killmails don't Generate

This is just going to be a quick technical explanation on why, simplified as much as possible, but no more.

This is a killmail in its raw format as provided by the CREST API, specifically this killmail. Without any real technical knowledge, you can skim over it and pick out the the system name, the ship type, the time/date, and the attackers on it. If you scroll down, there's a large section near the bottom that shows all of the items stored in the ship. The "flag" value tell you where the item is on the ship. For example 11 = the first low slot, 23 = the 5th mid slot, 90 = the SMA and so on.

Here's a shorter killmail highlighted with specific information:

However CREST being the new fangled toy, it uses the more modern JSON format for this data. Internally, EVE stores killmails in XML format and this is also the format Zkillboard pulls in new killmails from your API with. Here is the same shorter killmail as above pulled directly from my API in XML format. It contains exactly the same information but is much shorter since XML is more character efficient, and the CREST API is designed with ease-of-use for third party developers more than space efficiency in mind. 6,531 characters vs 1,994. No one really knows if that's exactly the format it's stored as internally in EVE's servers, but at a guess it's likely the same with some info excerpted (such as the itemIDs of the items on the killmail).


So now we look at the database. We know that EVE uses Microsoft SQL Server at its core as this has been stated many times before in various technical literature and dev blogs. SQL or "Structured Query Language" is a language for "asking questions" of the database. The database is a set of tables that you can think of much like Excel sheets. For each table, you define a set of columns, their names and their "types", i.e. "text", "integer", "decimal", etc. Once you've done that, you can then define the "relationships" between the tables. For example, here's the relationship diagram for the tables in Capri Auth:

If you notice on that the type "varchar(x)" is quite common. varchar is a type that just means "text of variable length" with x being the maximum length. For example, the maximum valid length for an address is 256 characters, but most are 20-40 characters, so varchar(256) is perfect.


This is where things go wrong. When the Nalvula Keepstar failed to generate a Killmail, a CCP employee explained on tweetfleet that it was due to an insufficient character limit somewhere in the database for an XML blob. The exact details are entirely speculation for those of us not working at CCP, but the running theory is that either the "attackers" part of the killmail or the whole thing is stored in XML format in one of these varchar(x) columns. If you get enough people on the killmail, the following likely happens:

  • Thing blows up
  • The struggling node slowly generates the killmail
  • Generation finishes
  • Node tries to save the killmail in XML format to the Database
  • Database: "Error, your entry for column [killmail] exceeds the character limit, truncating data"
  • The node quietly accepts the error and deletes the data from it's memory
  • The killmail doesn't pop up anywhere in game or on the API because randomly cutting a bunch of XML off the end essentially corrupts it.

After the Nalvula fight, someone was probably able to go in by hand and piece together enough of the data to make a killmail that got as many people as possible on it without hitting the character limit.

So what's the takeaway here? Well for a start, a whole bunch of what I just could be and probably is wrong. As a programmer, you tend to work back from the starting point "what would I do in their shoes", but often you're wrong about what tools they have at their disposal and no two programmers will ever come up with exactly the same solution to exactly the same problem. However the following is true: a killmail failing to generate isn't just a glitch due to an overloaded node, but an unfortunate and guaranteed side effect of having over a certain number of people on the mail.

To my knowledge, the main recent instances of this are the Nalvula Keepstar and the charity Revenant last year. Unless CCP have changed something major in the last few months, it's virtually certain we won't see a killmail come out of the fight on Saturday.