. Torchwood Archives
Torchwood Archives

LMeve dev blog: drill down & aggregate

Hi space friends!

Today I'm going to talk about how important constructive feedback is and how to show more information in the same amount of space 🙂

A few days ago I was contacted by an interpid industrialist who is using LMeve in his daily work. The amount of ideas this pilot sent me was amazing - thank you very much, sir! There were also two bug reports in that single mail. Awesome!

This is the kind of feedback is very useful, so if you have ideas for LMeve or found a bug in it, please don't hesitate to contact me.

Okay, now onto the new features

First off, the Industry Statistics got some love, namely, I broken down the amount of jobs started by activity type. So now you see how many of them are manufacturing jobs, and what else did your corp members do. The underlying code has been changed as well, because previously I constructed the JSON data for graph plugin by hand (inefficient!) and now I create a PHP object that gets json_encode()-d instead. In the end, more information is displayed in the same amount of space, and the code is less spaghetti!

industry-statisticsThere will be one more graph coming in Industry Statistics - with jobs delivered this time.

The second feature was requested by our new CEO (hi Razeu o7). In short, it allows to aggregate the Timesheet output at player level rather than character level. Some corp members use all of their characters for industry, and paying out monthly wage to each character is very inefficient - plus those players will then have to move the ISK to one pilot afterwards if they want those hard-earned money in one wallet. So now instead of a list of characters and how much they earned, you get a shorter list made of players.


If you wonder if running high sec industry CAN earn you a PLEX  monthly, wonder no more. One of our players made 837 million ISK last month. He was only running industry on two characters, and one of the toons made over 600 million! Like that kind of income? <recruiter hat on> Apply to Aideron Technologies right away! </recruiter hat off>

The last new feature is a little thing (now I know how CCP Karkur & CCP Punkturis feel when adding little things), but for some players it will proove very useful: EVE Time is now displayed on the top of LMeve window. It is a client side thing, so your computer clock has to be set right for this to work (enable NTP, set the correct time zone).

Bug hunting is a thing these days

The second part of the last LMeve update is bug fixes. There are three in this release:

  • Session timer now correctly counts since LAST ACTION done in the app, rather than since logon time
  • Ore Value Chart is now restored (a small bug related to code refactoring prevented it from displaying)
  • CREST is now using HTTPS. For some reason I forgot to do this earlier and it was using HTTP. Doh!

As usual, the latest update is available on Github.


SKINs support added in LMeve (with CCP WebGL preview)

Hi space friends!

Team Size Matters at CCP has recently shipped a very cool feature that I've already praised in this post. In short, you can now apply different paintjobs to your ships, and you can even change them mid-flight, in space!

The first time I saw it in game I realized it would be damn cool to use CCP WebGL and have it in LMeve as well - either for reference, or to choose which SKINs you'd like to buy.

Steve Ronuken again proved his place on the CSM is one well deserved. He was so helpful, that he already imported the new SKIN YAML files to his MySQL conversion of EVE Static Data.

All I had to do is bind them to Database module in LMeve. Sounds easy, right?

Well, not so fast. While CCP was generally very helpful by providing all the data I need, they provided this data in a bit inconsistent way. I'm talking about SKIN icons in Image Export Collection. Every typeID in the game, be it a ship, module or even DUST weaponry has two sizes of icons in Types.zip file. It applies to all items, but not SKINs! SKINs have their icons in Icons.zip "because SKINs are not linked to typeIDs" according to CCP FoxFour. Well, I agree to some extent, but then when you look at it, all SKIN Licenses typeIDs have an icon in Types.zip already - they just have this exclamation mark icon instead of the pretty hull colour preview known from the game client.

I didn't realize that these icons were in the other file - as soon as I saw the exclamation mark, instead of digging in more files, I have immediately set out to create a piece of code to draw these icons myself.

It didn't take long before I ended up with this JavaScript piece: https://jsfiddle.net/zbasb4jc/

Possible? Possible! It does not look *exactly* like SKIN icon in game. First I didn't know how to do gradients of the top of my head, and secondly, by now you've realized the first icon above is for Wiyrkomi SKIN, right? Then it means it is good enough 😉

The second step was to modify the Database view to display a list of available SKINs for selected ship, and modifying code handling CCP WebGL to load the correctly skinned version.

Here's an example page for Caldari Raven, and a fullscreen 3D preview:

lmeve-skins-database lmeve-skins-preview

Feel free to explore this new feature yourself - the public version of LMeve Database has been updated to show SKINs as well!


Positive Feedback

I don't usually fly Amarr. When I first began playing EVE ten years ago, I've chosen Gallente for my main character. Gallente felt most European from all the races, and I really didn't like the slavery thing in Amarr background. Caldari reminded me too much about rat race, and I was afraid the Minmatar ships will break itself apart. Let's say I didn't trust in the rust back then.

But now thanks to the new SKIN system I began to fly Amarr. Why? Because Khanid Oracle is simply gorgeous.


Ship skins and customization isn't a new idea. I mused about flying Gurista Raven back in 2012, and T'Amber has been developing his ship painting tools for at least a couple of years. So what makes ship skinning such a spot-on feature? It's microtransactions after all, and players didn't seem to like them back in Incarna.

There's a couple of reasons. Incarna failed, because avatar customization in a spaceship game was not really meaningful for the players. EVE is a spaceship game, set in space, so SHIP customization feels way more fitting. Other developers understood it earlier. In WoW the main way of transportation are mounts. Thus mounts make an important element of core gameplay. There are premium mounts, which, except for different looks, work exactly the same as their regular counterparts. The core gameplay of World of Tanks is... surprise: tanks. Tanks can have different camo patterns that can be purchased for in game money (time limited) or real money (permanent). Why do people buy them? Because they can customize element of the core gameplay (well camo reduces tank visibility, so it has actual in game function, too). And now EVE has ship skins. Fits just right!

So why do I like Khanid Oracle? As I said earlier, I didn't like Amarr. Khanid Kingdom is part of Amarr Empire, but they are closer to Caldari, and I can live with flying Caldari ships (because Gallente should know their nemesis). That's one of the reasons I really liked flying my brick tanked Damnation 🙂 Ship skins are not just a way to customize ships, for me it actually opens up a whole race I've been ignoring for the past 10 years.

So my TL;DR feedback for CCP FoxFour and the whole Team Size Matters is: SKINS ARE GREAT, I WANT MORE! 🙂

What next?

Well, some skins could have an actual in-game meaning or mechanic tied to them. For example pirate ship skins, when activated, would cause NPCs of the same faction to ignore the player's ship. But should the player engage these NPCs, the skin would expire immediately and rats would start shooting. This is of course just one example, but with working skin system, the possibilities are there.

*) The above idea is intended only for 30-day pirate NPC skins which drop as loot.


Site restored following a VMware crash

In the middle of the night last week, Torchwood Archives, LMeve Database, Yearly Stats and Aideron Technologies instance of LMeve vanished from the internet.

But it wasn't a DDOS attack.

It turns out hypervisors (such as VMware) sometimes crash, just like Windows does.

There was connection to neither the Linux system that powers my websites, nor to the VMware running under the hood. So at first it all looked like a good old fashioned hardware fault. It took a good while to restore, because my server is colocated in a different city, and due to real life I couldn't just go there and investigate it locally.

Here's where a remote KVM comes in handy. It's a special device that acts as keyboard, mouse and monitor, but you can access it remotely by visiting a special web page.
KVM which was connected to my server died a week earlier and was sent to RMA...

With connectivity to Linux and VMware gone, and KVM unavailable, the only way was to have a look locally, by connecting keyboard and monitor directly to the server.

What did the datacenter techie see on my box? Bluescreen I hear you say. Close. But on VMware bluescreens are... Pink. So there was a Pinkscreen waiting there. A simple power restart was enough to restore all the services back online.20121120_124732093_iOS

P.S. My coleague from work says VMware Pinkscreens are not PINK, they are PURPLE.


LMeve dev-blog: Bugfixes and POS tracking improvements

Hi space friends!

More and more of you have already tried LMeve, which I am incredibly grateful for. This means more people are testing the app and I get more feedback, which is awesome. I also get a lot of questions which I try to answer the best I can and in a timely fashion.

Remember: by trying and using LMeve you help developing it!

Few players also caught some bugs, that I'm happy to announce, have been properly squashed in the 0.1.52 release.

  • Bug fix in db.php "Notice: LM_dbhost is not defined" - added $LM_dbhost to global declaration in db.php
  • Bug fix in Database - "function eregi() is deprecated" - replaced with preg_match()
  • Bug fix in YAML updater - changed 'DROP TABLE' to 'DROP TABLE IF EXISTS' to prevent errors if the table was not created yet.

One bug was found by my trusty Aideron Technologies beta testers:

  • Non-recurring Tasks (singleton) bug fix - one-time tasks did not display correctly due to a mistake in SQL query

What's new besides Bugfixes?

  • Poller now fetches StarbaseDetails.xml and CREST industry cost indexes

The first API feed is used by POS tracking code to display accurate and up-to-date fuel levels in LMeve Inventory. The second CREST feed provides NPC cost index to the build cost estimation formula (I currently use an average system index, so you might pay more or less than what LMeve uses for price estimates).

In the next release I will add a "Industry home system" field in Settings, where you will choose your home system. LMeve will then use this system's cost index in all it's calculations.

  • New POS management screen - displays fuel amounts and how long it will last
  • There is now a notification when any tower has less than 48 hours of fuel left


  • Tech III invention added. You can now add Tech III invention tasks and get price estimates for Tech III
  • New global setting for the default Relic type for use with Tech III Invention (Intact, Malfunctioning or Wrecked)

Download LMeve here


If you are updating from a previous release, please remember to create new tables included in this release using /data/schema-delta-0.1.51-0.1.52.sql


Well, LMeve is free software, so you don't have to buy it 🙂 (donations are of course welcome!). But let's get back to business. Installation requires some technical knowledge, so to make it easier for you guys to try it out, I have prepared a pre-installed Virtual Machine with LMeve, which runs on free VMware Player. You can download the image here: https://pozniak.pl/lmeve-vm/ 

  • it contains LMeve 0.1.52
  • static data is updated to Scylla
  • after booting it shows the IP address to point your web browser to:



Yearly aggregated stats CREST endpoint coming soon

Right now in Reykjavik CCP Quant and CCP FoxFour are in the middle of a presentation, and they are showing a new CREST endpoint, which returns aggregate statistic data for your characters. For example, you will now be able to see how much damage your toon dealt last year, and how that compares to a year before! Of course this applies to a range of possible metrics, including, but not limited to:

  • number of ships built
  • number of BPOs researched
  • number of pod kills in wormhole space
  • number of characters typed in chat
  • number of stargate jumps in high security space
  • damage your character dealt with projectile weapons
  • number of times your toon was tackled
  • many, many more

Ready to see how you did in 2014?

Just open this page and follow instructions: https://pozniak.pl/yearly-stats (by the way this app is mobile device friendly).

slack_for_ios_upload_1024 Screenshot_2015-03-18-14-20-25IMG_1818 IMG_1820

Feel free to check Bellatroix and Aaeriele projects as well. They have pretty graphs!

Technical details

A few weeks before Fanfest 2015, CCP FoxFour asked on forums if there are any third party devs who would be willing to work on a "secret project". Few familiar faces turned up, and after some time three web apps have been built:


Unfortunately due to RL I was only able to work on my prototype app for several hours, but it has the core functionality of showing the most interesting statistics in a readable way 😉

Now speaking about the technical details, this new endpoint will be available as authenticated CREST, which means you have to use SSO mechanism first to obtain a Bearer Token (see this news item about authenticated CREST for reference). Only then it is possible to access these new endpoint on the CREST API server. Data itself is returned as a JSON object, so using it in JavaScript and PHP is trivial.

The URL where the data is stored looks like this:


where CREST_API_SERVER is either api-sisi.testeveonline.com or api.eveonline.com,

characterID - is the ID of the character (obtained from SSO process)

year - is either 2013, 2014 or 2015 (only these three years are available)

If everything goes fine, you will get a JSON file like this one:

{"combatDeathsPodTotal": 0,
 "combatDamageFromPlayersFighterMissileNumShots": 0,
"combatWebifyingPC": 0,
"socialAddedAsContactNeutral": 0

Once you have the data it's up to you how you want to display them.


CCP WebGL updated!

This is probably one of the best news for T'Amber and me in a long time: a CCP dev Filipp (we don't know his CCP nickname yet) has updated CCP WebGL library to use V4 shaders, and moreover he did a complete model export up to date with EVE Tiamat release. What this means is that third-party projects like LMeve and Jeremy can again display all ships in EVE in full glory with up to date shaders and models!

The new webGL library is a little different from the previous one, because it is using the new SOF mechanism (Space Object Factory) instead of plain .red files.

For example, to load an Onyx heavy dictor, old javascript call looked like this:

var ship = scene.loadShip('res:/dx9/model/ship/caldari/Cruiser/CC2/Kaalakiota/CC2_T2a_Kaalakiota.red', undefined);

and new call is a little different:

var ship = scene.loadShip('cc2_t2a:kaalakiota:caldari', undefined);

You probably noticed that instead of a file name we have three "ship DNA" strings now. The mapping between ship's typeID and the corresponding SOF entries are stored in graphicIDs.yaml, which is available in Static Data Export:

graphicID description sofFactionName sofHullName sofRaceName
38 Caldari Frigate Bantam caldaribase cf1_t1 caldari
39 Caldari Frigate Condor caldaribase cf2_t1 caldari
40 Caldari Frigate Griffin caldaribase cf4_t1 caldari
41 Caldari Cruiser Osprey caldaribase cc1_t1 caldari
42 Caldari Cruiser Caracal caldaribase cc3_t1 caldari
43 Caldari Battleship Raven caldaribase cb1_t1 caldari
44 Minmatar Frigate Slasher minmatarbase mf1_t1 minmatar

I have already updated LMeve to use the new library, but because of a small issue (more on that below), it currently only works if your LMeve is served using HTTP.

Updated Onyx model

Public LMeve Database has been updated as well. Here's an example page with the Amarr Confessor (please note you must press "3D" button for the WebGL preview to load).

All roses have thorns

Unfortunately the current version of the CCP webGL library has some drawbacks, too:

  • there are two model/texture sources now (so a simple reverse proxy like the one I've developed for LMeve won't handle it)
  • both sources use HTTP, so modern browsers like Chrome will trigger a security error when trying to load HTTP based resources from HTTPS based website.

The latter is a very simple thing and I'm sure Filipp will be able to correct it. With CORS headers now properly set on CCP CDN we no longer need a proxy to serve models and textures.

New version of Jeremy in the making

With the new webGL library T'Amber sees a lot of potential to improve Jeremy. the new version would be using dynamically generated list of ships (instead of hardcoded list maintained by T'Amber). I will help with all the backend work (for example serving all SOF and invTypes data by JSON API).

Here's a little teaser:

Stay tuned, because we are working hard to deliver you the best possible out of game ship spinning experience 🙂

AVvDxvj cpj6GjW


Victorieux Luxury Yacht

A few days ago CCP Gargant has announced the first inter-server EVE Online Tournament. See, there are no shards in EVE Online, there is only one server for the whole world: the Tranquility. Well, almost. There is one more server with a slightly modified version of EVE that complies to Chinese law and standards - Serenity.

Worlds Collide Tournament will pit the best PVP players from both servers against each other. But my post is not about the tournament itself, it's about the prize. See, the winning team will win Victorieux Luxury Yacht for every active player on their home server. This ship will never appear on the losing side's server ever!

Right now it's hard to tell if the ship will be at all useful (like the Gnosis), or will become just another collectible shuttle. It is however worth mentioning, that it is built on the model of Opux Luxury Yacht, which you could actually fit with modules (5 high, 3 medium, 6 low, 2 turret, 2 launchers, 50m3 dronebay). Opux' EHP is rather low when compared to rebalanced cruisers (it's on par with the pre-rebalance Exequror). This way or another, the ship has a beautiful model, and I totally understand why CCP wants to put it to a wider use.

I have faith in the CAMEL Empire and hope they will secure the ship for Tranquility (go go Camel Empire!), but before they do, we can give this glorious ship a spin in T'Amber's paitning tool Jeremy:

http://pozniak.pl/caldariprimeponyclub.com/?ship=Opux Luxury Yacht Intaki Syndicate Edition

Victorieux Luxury Yacht


Victorieux Luxury Yacht stats are now available in LMeve Database:


50% bonus to warp speed and warp acceleration

Immune to Interdiction Sphere Launcher, Warp Disruption Field Generator and Mobile Small, Medium and Large Warp Disruptor

100% reduction in Cloaking Devices CPU requirement

Can fit Covert Ops Cloaking Device


Developed for use by station governors and other well-connected members of the Intaki Syndicate, Victorieux-class Luxury Yachts combine opulent accommodations with state of the art cloaking and propulsion systems. The Victorieux enables comfortable and discreet travel for business or pleasure, from Poitot to Intaki and beyond.

The Syndicate has steadfastly refused to answer inquiries on how they obtained the technology necessary to enable simultaneous cloak and warp drive activation on this yacht.

In early YC117, Silphy en Diabel announced that the Syndicate would be sponsoring an invitational tournament and making limited run blueprints for the Victorieux available to all capsuleer supporters of the winning team.


LMeve dev blog: important fix in dates, proxy and one more thing

Hi space-friends!

I won't lie to you, I haven't been playing much EVE lately. All the spare time that I was able to find was used to improve and fix LMeve. And Aideron Technologies found a big-badda-bug this time.

On the last saturday of January, a corpmate asked me why he can't see the effect of his fire sales on the corp wallet graph in LMeve. It was the last day of January, and all graphs showed an empty bar for January 31st.

First I thought that poller might have stopped and we don't get up to date API data. But I was quickly proven wrong, because I could clearly see all the recent market transactions in the database. "Must be a mistake in code that feeds database values to the graph" I thought. But when I checked that code, it only confirmed what I saw on the graph: that the value for the last day is indeed equal zero.

If the data is in the database, then it must be some kind of a presentation error; there is simply no way last day should show zero!

Let's have a look at the database query used to retrieve the wallet data for a specific month. The clue must be there!

SELECT SUM(awj.amount)/1000000 AS income,date_format(awj.date, '%e') AS day FROM
apiwalletjournal awj
JOIN apireftypes art
ON awj.refTypeID=art.refTypeID
WHERE awj.date BETWEEN '${year}-${month}-01' AND LAST_DAY('${year}-${month}-01')
AND awj.corporationID=${corp['corporationID']}
AND awj.refTypeID <> 37
AND awj.amount > 0
GROUP BY date_format(awj.date, '%e')
ORDER BY date_format(awj.date, '%e');

Everything looks good from here, we SUM all transactions with positive "amount" field, and group them by the "day". Of course I want output in millions of ISK, so I divide it by 1 000 000. Then we only select records between '$year-$month-01' and the last day of the same month. Everything else is just filtering the correct corporation (because one instance of LMeve can monitor many corps at a time), filtering out player donations (refTypeID 37) and ordering the output by day, so we get a list of values for an entire month, from 1st day till the last.

How come this query will always return zero for the last day? Maybe that LAST_DAY() function actually returns last-but-one day?

SELECT LAST_DAY('2015-01-01');
one row selected

Hmmm, it works exactly as advertized...

Or does it? Records in the wallet journal table don't just have a DATE of the transaction, they have a full DATETIME! And what LAST_DAY() returns, is midnight on the last day of month! The entire timespan from midnight of that day until 23:59 that day is... well... filtered out. That explains why the last day was always missing.

How to fix it then?

Well, since we're missing exactly 24 hours, it's as simple as adding 1 day interval to the output of LAST_DAY():

WHERE awj.date BETWEEN '${year}-${month}-01' AND DATE_ADD(LAST_DAY('${year}-${month}-01'), INTERVAL 1 day)

LMeve CDN proxy

Now something else. As you know, I am now hosting T'Ambers ship painting tool - caldariprimeponyclub.com. In short, the problem was with the CCP CDN not setting Access-Control-Allow-Origin * in the header for users in AU and NZ regions (or some transparent proxy on the way stripped that header). Regardless, the end result was that T'Amber's tool was unable to fetch 3D models and textures and ended up with a blank screen. I had a similar issue in LMeve a while back, and I made a very simple proxy in PHP, which would fetch the file from CCP CDN using curl, add the necessary CORS headers, and then send it back to the user.

The original proxy was really crude and could cause unnecessary network traffic, because every time user asked for file 'A.png' I would download file 'A.png' from CCP CDN, even if I have downloaded this file five seconds ago! Couldn't I just save the file on disk, so I only retrieve each file once? Well, I've added the necessary logic, and the file is now saved in a cache, so I only retrieve each and every file once.

Now the second thing we needed was Analytics. It is a simple thing really: it's statistics and visibility. I keep a log that contains information about the file (filename, path and its size in bytes) if the file was served from my local cache or retrieved from CCP servers, when it was retrieved, and the IP address of the client (which makes pretty much a standard proxy log). Once I have all this information, I can plot pretty graphs showing how many unique users were there on any given day, how many files they retrieved, and how much traffic they created:


Happy ship painting!


One more thing

As you have noticed on that last screenshot, LMeve has a brand new skin. That skin is not just a random beige palette. I recently learned of a color scheme which improves readability in text and programming editors, dubbed "Solarized". That colour set contains 8 monotones and 8 accent colours which have several unique properties. First, they cause much less stress on the eyes, and second, they are displayed correctly on old and new displays, even on intentionally miscalibrated ones.

These two new skins have been added under Settings -> Preferences: solarized-light.css (above) and solarized-dark.css (best for night owls). They are way more minimalistic than original LMeve skins, and they don't contain too many images, which makes them "work-safe".


Hint: you can go further and remove all the images from LMeve. To do that, simply make a copy of your favourite skin .css file (they are in /wwwroot/css/ directory) and add this code:

img {
visibility: hidden;

This will hide all the images (character portraits, item icons and so on), but will retain the space that these images would normally occupy. This way LMeve remains completely usable, but becomes a "text only" application, that looks like everything but game 😉


Torchwood Archives is now hosting caldariprimeponyclub.com

Hi spacefriends!

A lot of you probably know T'Amber's ship painting project, that used to be available at caldariprimeponyclub.com.


Well, our friend T'Amber had some hard time making it work with models and textures hosted on the CCP CDN (for some reason CORS headers were set for some people, but were missing for others, so Jeremy was not able to load texture files).

But there is no such thing as impossible for two dedicated capsuleers 😀

Together we've been able to restore Jeremy's core functionalities by moving it to my host, and configuring it to use LMeve's built in proxy for model and texture files originally hosted by CCP CDN.

To access Jeremy, simply head to http://pozniak.pl/caldariprimeponyclub.com/