About two weeks ago I have started a new project, which uses XML feeds from EVE API. The new app is supposed to be a new industry tracker for my corp, because the old one is not being maintained anymore. We could probably use DRK Industry Tracker for that, but our Directorship has raised some doubts regarding security, because our API information could potentially be available to a third party. A decision was made to develop an in-house solution.
I have already written an EVE API app in Java to view character data in the past, so I have volunteered to be the chief architect and developer of the new app, thinking that API holds no secrets from me. Of course I was very wrong, but I will get to that later.
An industry planner app will have several users, who need to have access to it. This forces either a client-server approach (which would be an overkill for an app like this) or web based approach. Since browser applications only run when they are being visited, I would need some way to keep API data complete and without any gaps.
That’s how the API poller was born
I needed to write a poller script, which could be run in crontab periodically. This way I could keep the live data complete and up-to-date, without the need for anyone to trigger the API update (by visiting app’s web page for example).
EVE API has some best practices, one of them being respecting cache timers (to prevent unnecessary load on API servers). Another one is not to poll a feed that returned a persistent error (for example permission denied). I took these two as the main guidelines for my API poller.
It takes some experimenting
I’ve mentioned that I was wrong to say that I know the EVE API well. Well, I was. According to the documentation on the official EVE Wiki, WalletTransactions.xml.aspx only takes three parameters:
keyID, vCode, characterID
It means that the actual call should look like this:
https://api.eveonline.com/corp/WalletTransactions.xml.aspx?keyID=12345&vCode=1a23f91d20c&characterID=12341234
That call will succeed. You will not get an error message, but a parseable XML file with no data in it.
After some research it turns out one needs to provide the accountKey, which corresponds to wallet division. Then and only then will the WalletTransactions feed return the actual data. So the correct call will be:
https://api.eveonline.com/corp/WalletTransactions.xml.aspx?keyID=12345&vCode=1a23f91d20c&accountKey=1001
Getting the data is not even half the trick
The EVE Static Data Dump is about 300 megabytes of data. The live information downloaded by the poller adds even more data to dig through. This means that the most important thing is analyzing, displaying and managing all this data. GUI has therefore several tasks:
- authenticate users
- authorize users
- allow corp members to self-register in a secure manner
- plan the industry tasks
- track the industry tasks
- run reports
- view the static data dump information
- enrich live data by connecting it with static data
- manage settings and API keys
While it sounds simple, the majority of the work will be done here. Each of these bullet points will be broken into even more actions. Working application however, will help our corp make billions, so all the effort is well worth it.
