Category: Programming

Posts related to programming

LMeve state: almost done, still work in progress

Lukas Rox EVE Online, Programming , , , ,
LMeve state: almost done, still work in progress

Hi Spacefriends!

In the recent couple of weeks I’ve put in considerable amount of effort into LMeve development, in order to bring back all lost functionalities.

At the beginning of September I brought back the core functionality of LMeve, namely Industry Tracking. Since then the following functions have been restored:

  • added Markets route
  • pulls Jita prices from /markets//orders/ (default: Jita/The Forge)
  • pulls game calculated average and adjusted prices for each typeID
  • removed eve-central.com – all data comes solely from ESI now
  • added Corporation Market Orders
  • added Corporation Contracts and Contract Items
  • Buyback feature works again
  • Wallet Balance
  • Wallet Journal
  • Wallet Transactions
  • added ‘KIT’ endpoint in LMeve Northbound API. Usage: api.php?key=&endpoint=KIT
  • added XML format – use ‘output=xml’ and ‘output=json’ in URL to select format
  • Assets
  • Industry System Indices support added
  • Killmail support added
  • Adjusted killmail view ( killmails are now ESI verified 🙂 )
  • Killboard is back up!
  • Corp Blueprint support added
  • ME/TE values are based on actual Corp Blueprints

Additional changes:

  • fixed a bug in Universe route
  • moved ESI settings to GUI
  • added ability to use data from Sisi (support for ESI datasource)
  • GUI tuning for Buyback feature (ESI has new contract states)
  • Deprecated and disabled everything that mentions old XML API
  • Bug fix in X-Pages support
  • Twitter card meta tags for Killboard – links look nice when sharing killmails

Stuff remaining to be done

  • Customs Offices
  • Citadels
  • Industry Facilities
  • Character connect using SSO
  • Member self registration using SSO (optional, disabled by default)
  • Bug fixing!

Known bugs

  • Manufacturing Price estimation does not compute correct prices despite having all prices in the database

LMeve state: back from the dead, work in progress

Lukas Rox EVE Online, Programming, Software , , , ,
LMeve state: back from the dead, work in progress

Hi Spacefriends!

First of all, thank you for continued support and all your e-mails and EVE mails. I was surprised that so many people still use LMeve! I can’t simply ignore so many kind words and encouragement. Thank you for that!

I’m proud to announce that despite limited free time, I’ve finally managed to restore the core functionality of LMeve: Industry Tracking.

LMeve Industry tracking using ESI LMeve Industry tracking using ESI

User Story

You might wonder why it took so much time to restore any functionality, so here’s the story. When XML API cutover was announced, I realized I have to modify the backend of LMeve: the engine that handles API and feeds LMeve with data. But the old backend was written in plain procedural PHP. It was spaghetti-quality code, that was hard to modify and maintain, and even the simplest modification had to be done in multiple places in code. So I decided I have to rewrite it from scratch, this time as well as best as I could, so the new code would be easier to maintain once it’s done. The new ESI backend is written in objective PHP, with two main template classes and several inherited classes. The main one is ESI class and it represents the API itself. It allows access to routes, handles refresh and bearer tokens and all the necessary operations such as token renewal. You can easily access any route using simple but powerful instantiation: $ESI->Characters->getCharacterName($characterID) for example will provide a toon name when provided with character ID. Very easy to use, right? Each route inherits the Route base class, which handles the low level network communication, reporting and error handling. It also supports paginated output on ESI routes using the X-Pages header.

Moreover, this class can be used not just in the backend of LMeve, but in the GUI as well. It can be used to fetch data from ESI directly. Other apps such as my Annual Stats can (and actually already do) use it. This is why it was worth to take the rewrite-from-scratch approach, even if it took more time!

How to upgrade

  1. Right now simply download the new PHP code and overwrite your current installation. Don’t worry about the database tables, because DB will be updated automatically the first time you log in.
  2. Second step is to enable the EVE SSO using this helpful wiki guide: Integrating LMeve with EVE SSO .
  3. Then go to Settings and make sure Enable ESI Poller is checked.
  4. Last but not least, head to Settings -> ESI API tokens and create New ESI token. Please mind you must have a Director role to do this.

What works, what does not

Please remember this newest version is still a beta. Logging is heavy, so you might want to rotate your logs more often.

What works:

  • ESI integration and token management
  • ESI status checking and alerting in case of failure
  • Corp Member information
  • Industry Tracking and Tasks Tracking

Roadmap for the next releases:

  • Market prices
  • Assets
  • Wallet
  • Contracts
  • Killboard

I will release one feature at a time, using Agile approach.

What can You do to help?

I would like to ask You for some support. How can You help?

  • Use and test the new LMeve version. Bug report on the project’s Github page (please check the list of features that currently work and which don’t before you report)
  • Need a new alt for Industry? Use my buddy invite to register. You will get an extra 250.000 SP on your new character as a bonus.
  • Wanna send a tip? I could always use some extra buck! Here’s my PayPal donation page. It will also be easier to convince my better half that I need to spend some time on a freeware app for video games 🙂 just sayin’

Insert coin!

Thanks! Your help and support is always welcome and appreciated!

Special thanks goes to: Random McNally, Rixx Javix, Ramdar Chinken, Marq Aideron and Starfire Dai.

LMeve state – ESI support is not ready

Lukas Rox EVE Online, Programming, Software , , ,
LMeve state – ESI support is not ready

Traffic advisory

I’ve started updating LMeve to ESI four months ago (see the project site on github https://github.com/roxlukas/lmeve/), but due to RL being rather harsh on me for the past couple of years regarding free time, I don’t have enough of it to finish necessary modifications before XML API cutover. I’m truly sorry about that, but there is little I can do to speed it up. Unfortunately I haven’t found any other third party dev interested in helping me update LMeve either.

Here’s the current status:

  • new ESI poller library: DONE
  • ESI base class (token exchange, cache handling, exception handling, performance measurement): DONE
  • GUI for authentication and token echange: DONE
  • corporation sheet ESI-to-database mapping: DONE
  • corporation members ESI-to-database mapping: DONE

As you can see a lot of the low-level stuff is already done, but the most important now to make LMeve work would be to continue mapping data from ESI to LMeve database tables:

  • industry jobs
  • contracts
  • killmails
  • wallet journals
  • market orders
  • market prices

I haven’t abandoned the project, but it has to wait until RL settles a bit (i.e. doing less projects in the evening). This may take a month, or this may take five months, I can’t tell. Maybe if LMeve *was* a commercial project, it would speed things up… but I really want to avoid running a kickstarter for this. I’d like to keep LMeve free for everyone to use.

TL; DR

LMeve will no longer receive data from EVE Online servers after downtime, May 8th, 2018.

LMeve dev-blog: Killboard with WebGL ship preview

Lukas Rox EVE Online, Games, Programming , , ,

It was quiet on the blog lately, but behind the scenes I was receiving tons of valuable feedback, and development was slowly but surely progressing through the past month.

There were two recurring questions in the feedback I was getting:

  • Corp API Keys editable in GUI
  • Killboard

So these two features have been made the top priorities. Here’s the results:

Killboard

Some people set up an instance of LMeve, and then on the same host they set up a killboard. How convenient would it be if one app contained both? Good idea. But there is so many killboards already available, that writing a new one which copies existing functionality made little sense. I had to do something new!

In the previous release, LMeve got ship fitting preview in Inventory, and it felt like a great way to display the fitting of destroyed ships as well. But first, I need the killmail data.

Killmails are available from two APIs: XML (with corp API Key) and CREST (you need to know certain information about the killmail to be able to fetch it). Item list in the XML API lacks specialized cargoholds, so I opted for a hybrid approach. First, I download Killmail.xml.aspx, based on this information I can calculate a killmail hash required by CREST, and finally download the CREST Killmail to fetch complete item list, including fittings and specialized cargoholds. This default behavior can however be changed in Settings, where you can choose to use plain XML API instead (it’s a little faster, but fittings of destroyed ships might be incomplete).

Right, but theres nothing new here, right? zKillboard does this out of the box. Well, true.

That’s where the second feature comes in. I changed the fitting display to include a 3D ship preview (rendered using CCP WebGL) instead of a static ship image (which, mind you, is a fallback for older PCs/browsers). Yup, you can spin and zoom the ship on the killmail! It works in Inventory, too!

Enough words, pics or it didn’t happen:

killboard webgl_killboard_4 webgl_killboard_3 webgl_killboard_2 webgl_killboard_1

Corp API Keys editable in GUI

This is a very simple, yet important feature. Previously, during LMeve installation procedure you had to input your API Keys in PhpMyAdmin, by directly inserting into a database. I’ve chosen not to expose API Keys in GUI, because they could potentially be abused. But this made already complicated installation procedure even worse, so after some deliberation I’ve decided to add a GUI for it.

corpapikey

And here’s the complete list of this month’s features, coming to LMeve in 0.1.55:

  • Fully functional Killboard
  • XML and CREST killmail support in API Poller
  • Can use either XML or CREST killmail endpoint (configurable in Settings)
  • CCP WebGL preview in Inventory and Killboard ship fitting window
  • You can now input Corp API Keys in GUI
  • Disabled accounts have existing sessions terminated immediately
  • Bug fixes

LMeve dev blog: Does inventory only track minerals?!

Lukas Rox EVE Online, Games, Programming , ,

This headline is actually one of the most often asked questions by brand new LMeve users. And it makes sense after all: the default install would only have minerals set to “Track stock”, so only minerals would show after opening inventory.

Well, not anymore.

Inventory has been rebuilt to resemble in-game inventory. All belongings of your corporation, be it POS structures, offices in stations or even containers – will now be available for browsing.

inventory

Pretty neat, eh? Okay. You’ve noticed those assembled Exequrors over there? Are you thinking the same thing I’m thinking? Wouldn’t it be cool to inspect the fitting of such an assembled ship?

It would!

When you hover on a fitted module, LMeve will show a tooltip with it’s name in a similar manner EVE game client would.

Ok. Now onto the next thing. As you most likely know, LMeve Database already sports 3D SKIN preview. I’ve refined this feature even further by giving it the ability to load any SKIN from the matching race, not just the ones you can buy from NeX store.

Quafe Exequror anyone?

quafe-exequror

Please mind this feature is readily available in the public LMeve Database, so go ahead and take a peek!

So many images and so little reading! Well, I had to show you the most iconic features of this release, because one picture is worth a thousand words they say. Wait, did I say iconic features?

This release actually has new support for item icons, and when you’re installing LMeve you can now choose to host the typeids icons with your instance (and update every 5 weeks as new EVE releases show up), or rely on CCP imageserver. Keeping your install up to date is one bit easier now!

There’s also a few smaller changes

  • Additional tooltips – should help new users understand how things work
  • Customizable market hub to get material prices from – defaults to Jita, but you can freely change it
  • Customizable manufacturing sytem Index (Crius NPC cost formula) – same as above, used to calculate Manufacturing costs
  • Customizable price modifier for Buy Calculator – if you want to pay more (or less) to your corp mates for their minerals, set the modifier here
  • Real time API poller stats – shows in real time what the API background process is doing at the moment
  • Bug fixes

That’s it for today, and as usual, the new release is up on Github: https://github.com/roxlukas/lmeve

Fly safe o/

Oh my, I’m a one man Support Center!

Lukas Rox EVE Online, Games, Programming

As some of you know, I spend more time on developing tools than on playing EVE. But development of tools is not just about programming.

When you open your application up for other people to use, you will start getting questions eventually:

lmeve-support-center

Yup, this is just one weekend of messages!

While I totally don’t mind that, and despite limited time, I try to get back to every single one of you as soon as I can, some of the questions repeat. So to avoid answering the same questions, I’ve created a FAQ on the project’s page on Github:

LMeve Frequently Asked Questions

I hope it answers at least some of your inquiries!

Fly safe o7

– Lukas Rox

PS. LMeve project wiki also holds Installation manual, End user manual and LMeve Northbound API manual

Aegis Static Data Dump Conversion Delayed

Lukas Rox EVE Online, Games, Programming , , ,

Update: Aegis SDE conversion is now available: https://www.fuzzwork.co.uk/dump/

Most of the players don’t realize that almost all third party apps need to be updated every time there is a new release of EVE. This is because of the Static Data – a special database made by CCP and released to the third party developers of EVE. It used to happen two to four times a year, but with the new release cadence it now happens every 5 weeks.

Usually the process is rather simple: CCP releases a MS SQL backup with the static data (plus some tables in YAML files), then Steve Ronuken (rather famous third party developer and a CSM member) combines both MS SQL and YAML files into one databse and converts it to several formats, such as MySQL (by far the most popular one), PostgreSQL and SQLite.

In Aegis release, CCP has changed one of the core tables – invTypes – from MS SQL database table to a YAML file. And unfortunately it caused some trouble:

Steve Ronuken [1:42 PM] It’s just because there’s data missing, due to the conversion

Steve Ronuken [1:50 PM] invGroup, invCategory data is totally missing. much of invTypes is missing

and

Forum thread about Aegis 1.0 SDE – Released

What does this mean for a regular EVE Online player? Most of the third party apps will be stuck on Carnyx static database, meaning there will be no Hecate or new missile modules there. We have to wait until CCP rectifies the situation, so Steve can convert the DB for the rest of third party devs.

Maybe it would be a good idea for CCP to consider releasing a complete database in all popular formats (or at least one), instead of data being scattered between MS SQL backup and several YAML files?

LMeve dev blog: drill down & aggregate

Lukas Rox EVE Online, Games, Programming , ,

Hi space friends!

Today I’m going to talk about how important constructive feedback is and how to show more information in the same amount of space 🙂

A few days ago I was contacted by an interpid industrialist who is using LMeve in his daily work. The amount of ideas this pilot sent me was amazing – thank you very much, sir! There were also two bug reports in that single mail. Awesome!

This is the kind of feedback is very useful, so if you have ideas for LMeve or found a bug in it, please don’t hesitate to contact me.

Okay, now onto the new features

First off, the Industry Statistics got some love, namely, I broken down the amount of jobs started by activity type. So now you see how many of them are manufacturing jobs, and what else did your corp members do. The underlying code has been changed as well, because previously I constructed the JSON data for graph plugin by hand (inefficient!) and now I create a PHP object that gets json_encode()-d instead. In the end, more information is displayed in the same amount of space, and the code is less spaghetti!

industry-statisticsThere will be one more graph coming in Industry Statistics – with jobs delivered this time.

The second feature was requested by our new CEO (hi Razeu o7). In short, it allows to aggregate the Timesheet output at player level rather than character level. Some corp members use all of their characters for industry, and paying out monthly wage to each character is very inefficient – plus those players will then have to move the ISK to one pilot afterwards if they want those hard-earned money in one wallet. So now instead of a list of characters and how much they earned, you get a shorter list made of players.

aggregate-by-player

If you wonder if running high sec industry CAN earn you a PLEX  monthly, wonder no more. One of our players made 837 million ISK last month. He was only running industry on two characters, and one of the toons made over 600 million! Like that kind of income? <recruiter hat on> Apply to Aideron Technologies right away! </recruiter hat off>

The last new feature is a little thing (now I know how CCP Karkur & CCP Punkturis feel when adding little things), but for some players it will proove very useful: EVE Time is now displayed on the top of LMeve window. It is a client side thing, so your computer clock has to be set right for this to work (enable NTP, set the correct time zone).

Bug hunting is a thing these days

The second part of the last LMeve update is bug fixes. There are three in this release:

  • Session timer now correctly counts since LAST ACTION done in the app, rather than since logon time
  • Ore Value Chart is now restored (a small bug related to code refactoring prevented it from displaying)
  • CREST is now using HTTPS. For some reason I forgot to do this earlier and it was using HTTP. Doh!

As usual, the latest update is available on Github.

SKINs support added in LMeve (with CCP WebGL preview)

Lukas Rox EVE Online, Games, Programming , , ,
SKINs support added in LMeve (with CCP WebGL preview)

Hi space friends!

Team Size Matters at CCP has recently shipped a very cool feature that I’ve already praised in this post. In short, you can now apply different paintjobs to your ships, and you can even change them mid-flight, in space!

The first time I saw it in game I realized it would be damn cool to use CCP WebGL and have it in LMeve as well – either for reference, or to choose which SKINs you’d like to buy.

Steve Ronuken again proved his place on the CSM is one well deserved. He was so helpful, that he already imported the new SKIN YAML files to his MySQL conversion of EVE Static Data.

All I had to do is bind them to Database module in LMeve. Sounds easy, right?

Well, not so fast. While CCP was generally very helpful by providing all the data I need, they provided this data in a bit inconsistent way. I’m talking about SKIN icons in Image Export Collection. Every typeID in the game, be it a ship, module or even DUST weaponry has two sizes of icons in Types.zip file. It applies to all items, but not SKINs! SKINs have their icons in Icons.zip “because SKINs are not linked to typeIDs” according to CCP FoxFour. Well, I agree to some extent, but then when you look at it, all SKIN Licenses typeIDs have an icon in Types.zip already – they just have this exclamation mark icon instead of the pretty hull colour preview known from the game client.

I didn’t realize that these icons were in the other file – as soon as I saw the exclamation mark, instead of digging in more files, I have immediately set out to create a piece of code to draw these icons myself.

It didn’t take long before I ended up with this JavaScript piece: https://jsfiddle.net/zbasb4jc/


Possible? Possible! It does not look *exactly* like SKIN icon in game. First I didn’t know how to do gradients of the top of my head, and secondly, by now you’ve realized the first icon above is for Wiyrkomi SKIN, right? Then it means it is good enough 😉

The second step was to modify the Database view to display a list of available SKINs for selected ship, and modifying code handling CCP WebGL to load the correctly skinned version.

Here’s an example page for Caldari Raven, and a fullscreen 3D preview:

lmeve-skins-database lmeve-skins-preview

Feel free to explore this new feature yourself – the public version of LMeve Database has been updated to show SKINs as well!
http://pozniak.pl/database/index.php?id=10&id2=1&nr=638