Open letter to FD - Please DO actively support third-party tools.

[wolverine2710]

Announcement: Saturday the 23th of January 2016.
Brilliant news: On the 5th of January 2016, Zac Antonaci -Head of Community Management of FD- wrote in the "Open Letter to Frontier Developments" thread that usage of the iPhone companion app api is LEGAL. Source. Which makes the usage of tools which use this api like EDMC, EDAPI and EDCE LEGAL. Other great news: On the 18th of January 2016, Brett C. -Community Assistant/Manager of FD- created the "Elite Dangerous->Gameplay and Features->Player Tools & API Development" subforum. Its an excellent place for checking out new and existing tools. The EDCodex thread is also there.

---

I'm writing this open letter in the sincere hope that we soon get (better) support from FD for third-party tools. Atm there are more then 100 tools in EDCodex.info - approx 210+ entries in total. I know for a fact that NOT all tools are in EDCodex. Active support by FD would mean an EXPLOSION of new and more varied tools.

In January Michael Brookes invited everyone to give functionality suggestions for an official ED api, see the thread "[PROPOSAL DISCUSSION] External API Requirements" Thread. On February the 19th 2015 Michael wrote it would take a few weeks to evaluate it all (source). On the 22th of May in the thread "Any news on the official ED API?" he wrote the following about the API: "It's still on our radar, but it is low on the current priority list. If anything changes we'll let you know. Michael" Source.

Atm there is little data for the third-pary tools they all centre around things like trading, logbook, shipyard. The community uses the following creative methods to retrieve data for the tools.

• OCR techniques. To extract market data from screenshots. The resulting .csv file can be imported in trading tools and market data is optionally sent to EDDN. Examples EliteOCR, Regulated Noise.
• Netlogs. When a debug flag is set in AppConfig.xml, tool authors can extract the current system a commander is docked in.
• iPhone web-api. This enables tool authors to retrieve market data, shipyard info, outfitting info and stats. They are also called OCR Error Free (OEF) tools. The three existing OEF tools: EDMC,EDAPI and EDCE) all emulate the official ED iPhone app released November 2014. EDMC stands for: Elite Dangerous Market Connector.
• Crowd sourced data. There is no official data about formulae used for for example fuel usage - needed for route planners. Shipyard tools have also resorted to crowd sourced data. Calculating 3D coordinates for a system is a pita as well. Especially after the GM distances have been reduced to 2 decimals.

The legal usage of the iPhone web-api is not 100% clear atm. An author of an OEF tool using the iPhone web-api asked if it would be allowed to temporarily use the iPhone web-api as long as the official External API has not surfaced, got the following answer from Michael Brookes on the 22th of May 2015: "It's being discussed. Michael" (source).

I can imagine that a full blown External API would take considerable time to create and test. What quite a few commanders have suggested in the past is the following as a good temporarily solution.

• Dump market data when you are docked at a station in a JSON/XML file.
• Dump the 3D coordinates of a system when you exit SC in a JSON/XML file.
• Dump docking status in a file.

This can be extended to ALL data which is visible on the screens. The data is already in the ED client otherwise it could NOT be displayed. Do this in the most simplified way possible. Just overwrite a JSON/XML file each and every time new data is available. This would be very easy to achieve. The community can write tools to extract that data and use it. Even the dumping of data could be made optional for each and every commander- setting in AppConfig.xml.

I personally think that ED would benefit hugely from a large and varied third-party tools ecosystem. When FD does not have the time to create ingame tools the community can pick up the gauntlet. An official anser/comment from FD about this would be highly appreciated.

Please do NOT turn this thread into a discussion about if you like or dislike third-party tools. There are quite a few threads where this has been discussed.What can be ofc discussed is what kind of data, now available on the screens/menus would be useful/essential for third-party tools.

To be in compliance with the FD rules I'm not supplying urls for OEF tool entries like Elite Dangerous Market Connector (EDMC) in EDCodex. A key feature of the website: Its heavily search engine optimized (SEO). Applying Google fu will be easy and effective.

 

[bootsam]

+1 Community involvement can only be a good thing. For what its worth, the above has my support.

 

[wolverine2710]

+1 Community involvement can only be a good thing. For what its worth, the above has my support.

Thanks for the post and the rep ;-)
Is their something which you would like to see in the suggested "dump" files what you consider invaluable for tools in general or for a specific tool?

 

[Agony_Aunt]

Just my personal view, but i'd rather FD spend their efforts to improve ingame tools so that third party tools are not required.

Of course, i'm not suggesting they implement a best trade route calculator, but make the ingame trading info more informative and useful (while less gameable ... is that a word? Make it less so people can just use tools to work out the best route, perhaps just make prices more volatile so traders have to move around more).

 

[Commander Wolzan]

I'm all for third party tools so this post has my support. Working on Elite Galaxy Online has been brilliant for me and if I could link it more into the game I could maybe do some pretty cool stuff.

 

[Otis B.]

Just my personal view, but i'd rather FD spend their efforts to improve ingame tools so that third party tools are not required.

In an ideal world, so would I. But people have been asking for in-game flight log, bookmarks, better exploring and trading tools etc since before 1.0 so 3rd-party tools are filling the gap.

Of course, i'm not suggesting they implement a best trade route calculator, but make the ingame trading info more informative and useful (while less gameable ... is that a word? Make it less so people can just use tools to work out the best route, perhaps just make prices more volatile so traders have to move around more).

I would quite like an in-game best trade route calculator, as long as it was limited to the information available or obtainable by you in-game. Couple this with making market data tradeable in-game and you broaden the game and open up new career options.

 

[wolverine2710]

Just my personal view, but i'd rather FD spend their efforts to improve ingame tools so that third party tools are not required.

Of course, i'm not suggesting they implement a best trade route calculator, but make the ingame trading info more informative and useful (while less gameable ... is that a word? Make it less so people can just use tools to work out the best route, perhaps just make prices more volatile so traders have to move around more).

I totally understand your point of view and respect that. Given what they currently have on their plate (1.5, horizons) and what they have done for in game tools since the release (16th December 2014) exactly 8 months ago I don't think we can expect something from FD for this (in game tools). The external API would have been brilliant but as I wrote "low priority" as stated by FD. I'm making a living as a software engineer myself and I can't imagine it would take them long to implement a "dump solution". That is the reason for this thread.

Wrt "moving around". In the so called YES and NO threads about third party tools it has been said multiple time that third party tools trading tools DO cause that commanders move around.

Atm approx 1200-1300 times a day EDDN receives market data (complete station/outpost price info is one message) from OCR and OEF tools. Which is peanuts and makes trading tools sub optimal useful. Some stats from Maddavo's website: Database currently holds 40821 Stations in 19010 Systems, 18414 Commodities Markets in 10458 Systems, 1013258 Prices. When a "dump" solution would exist it would allow for MUCH more data and it would allow for things like Kibana. What you can do with that is explained in "Economist Newsletter #1: Insight into the current market dynamics". Thread: "Economist Newsletter #2: The Eranin food crisis". Video: "[Video] Market data visualization with EMDN and ELK". With kibana you only get the data visualized and you still have to decide what to do. BUT for stuff like that to work we need much much much more data. A "dump" solution would also mean we would get much more variety in the third party tools.

 

[2plex]

Simply: the 'not invented here' law is being applied.

 

[Henri]

i'd like some way to interact with the universe from my phone, maybe buy and sell stuff or I don' t know...some companion app so I can do some elite stuff.

 

[Biobob]

In addition,
with the next 1.4 release, now some player groups will be ingame integrated, seems to be very nice,
Now waiting for release to see if we will have more explanations about that and to see how those new groups will interact with existing universe!
Seems we will have more community features incoming and a kind of API could be very useful: lot of major players groups got an external website and for sure it could be very interesting to be able having some live datas (group progression/rank, player's allegiance, war status, stats about average ranking/credits in a group...).

I know there is lot of work in progress with the 1.4, 1.5 & horizon but it could be nice to have some news about the official API (still planned?).

 

[kilvenny]

I think this is a really interesting challenge for FD.

My personal view is that FD should be opening up API's for areas that would allow the third party tool makers to add content that supplements the game rather than weakens it.

eg. A third party captains log. Provide information on location, missions taken, stations visited, kills, deaths, ship purchases and sales, rank changes.

I have tried the trading tools and much as I think they are great tools, I actually thing they weaken the game, because they provide a quick and easy way to find the grind sweet point without doing the graft. I stopped using one when I realised I was short cutting a whole load of game play. Normally what you would have to do is create your own list, visit stations and work it out for your self, as I remember this was a huge challenge in the original. If FD provide their own tool, the data will get sucked down into a third party tool for every one to reference. Of course for many it is what they crave.

 

[wolverine2710]

I think this is a really interesting challenge for FD.

My personal view is that FD should be opening up API's for areas that would allow the third party tool makers to add content that supplements the game rather than weakens it.

eg. A third party captains log. Provide information on location, missions taken, stations visited, kills, deaths, ship purchases and sales, rank changes.

I have tried the trading tools and much as I think they are great tools, I actually thing they weaken the game, because they provide a quick and easy way to find the grind sweet point without doing the graft. I stopped using one when I realised I was short cutting a whole load of game play. Normally what you would have to do is create your own list, visit stations and work it out for your self, as I remember this was a huge challenge in the original. If FD provide their own tool, the data will get sucked down into a third party tool for every one to reference. Of course for many it is what they crave.

A lot of the information you mentioned was already in the iPhone web-api. EDMC and EDAPI dumped it in a .csv file. It has been removed since 1.3.0.7 I believe. This could be made available in a "dump" file.

I agree that what one person calls weakening is what somebody else is graving for. Have a look at the kibana videos in post #7 to see what can be done when there is enough data. In that case you have to make decisions yourself based upon graphs. I'm sure similar third party tools will be created. Atm there is a very good ecosystem wrt distributing (dynamic) data. EDDN distributes dynamic data, tools like TGC and EDSM distribute or make available static data like 3D coordinates. There are atm 16 tools which use EDDN. If/when data will be made available it WILL be distributed and stored. Again, this is positive for one person and negative for another person. Personally I don't think it will hurt anyone if data is made available. Its a HUGE galaxy out there. 20K systems with multiple stations/outposts. Not sure but I thought that there will be stations/outposts in horizons to. Hence even more stations in total. I think its a miracle that the trading tools are so popular and work relative well if you look at with what data they are working with - most of the data is very old. 1200 messages a day in EDDN ;-( Still they work great, most of the time. It indicates to me that the market is not fluctuating that much. No real market mechanism at work. Imho when more data is available, commanders will spread out (even) more.

But that all is not the point of this thread. Its a polite request, asking FD to release a temporarily solution till the External API is ready. It would allow for an EXPLOSION of brilliant new tools created by the creative, very talented tool authors. See what they have done with basically zero support from FD. Imagine what can be achieved when there is plenty of different data.

 

[Khelder]

As someone who both uses third-party tools and who is writing one of his own, I wholeheartedly agree. I believe third-party tools are a real positive for the game. In many cases, I would prefer certain functionalities to remain the domain of third-party tools rather than being implemented in-game.

Personally, before data dumps and the like, I would really like just a vote of support from FD. Even if it's something along the lines of, "We are well aware of the vast range of third-party tools out there. We are encouraged to have such community enthusiasm. We have no intention of interfering with or attempting to block said tools."

As for what to have available, if FD actually went a step further. I think you've covered the main ones. I've got a couple of tricks I've been working on, but having basic data without the fuss would be handy. The big (and obvious - to me at least) ones that we don't have are complete docking status (docked status changed, current system, current station) and system coordinates (even if they supply accurate distances from known basepoints - it's better than guessing).

 

[wolverine2710]

As someone who both uses third-party tools and who is writing one of his own, I wholeheartedly agree. I believe third-party tools are a real positive for the game. In many cases, I would prefer certain functionalities to remain the domain of third-party tools rather than being implemented in-game.

Personally, before data dumps and the like, I would really like just a vote of support from FD. Even if it's something along the lines of, "We are well aware of the vast range of third-party tools out there. We are encouraged to have such community enthusiasm. We have no intention of interfering with or attempting to block said tools."

As for what to have available, if FD actually went a step further. I think you've covered the main ones. I've got a couple of tricks I've been working on, but having basic data without the fuss would be handy. The big (and obvious - to me at least) ones that we don't have are complete docking status (docked status changed, current system, current station) and system coordinates (even if they supply accurate distances from known basepoints - it's better than guessing).

Good points. I especially agree with the "fuss". Hundreds of human hours have been spent on algorithms to calculate 3D coordinates from distances. It got a LOT more difficult when distances were shown in 2 decimals instead of 3 decimals. Still the brilliant commanders made it work. I guess hundreds of hours have gone into creating good, decent (but still not 100% reliable) OCR, machine learning techniques. If all that time could have been spend on third party tools just using data made available by FD I'm pretty sure we would not have 100+ but 300+ tools. Its darwinism, good tools survive and bad tools go the way of the dinosaurs. Good and bad in the sense of what do commanders need.

Lets not forget, so far commanders have played nice with FD and didn't really hack the ED client. They easily could have done that. I've seen on IRC a zip file with 32K different files. All with brilliant but now ofc obsolete market data and much much more data like stats. No 3D coordinates though. A wet dream for tool authors. I'm also pretty sure that hacks ARE available, they just stay underground.

 

[Dommarraa]

I think the success of the 3rd party tools (including VoiceAttack) have made FD aware that there is money to be made here, even if they dont have a plan or resources to monertorize it yet. Providing an API now would scupper any plans in the future.

 

[wolverine2710]

I think the success of the 3rd party tools (including VoiceAttack) have made FD aware that there is money to be made here, even if they dont have a plan or resources to monetize it yet. Providing an API now would scupper any plans in the future.

Sadly, I've to admit I'm thinking something along the same lines - and I'm not alone in this. It has been brought up also in the "Any news on the official ED API ?" which is a very good read btw. In Elite: Dangerous Newsletter #66 section Licensing Elite: Dangerous commercial and non-commercial licensing possibilities has been mentioned. In the commercial license paid apps have been mentioned specifically. In post #18 of the "any news" thread the following was written:

I can imagine that FD wants to make sure that the paid apps authors are giving a change to make money - for them and of course FD. Those apps NEED an official ED API to work. If the BARE BONE (comment: "dump files" here in this thread) info would be there the free apps probably would steal their thunder - and make them obsolete or at least less attractive.

A few relevant questions comes to mind.

• Will the free apps and paid apps have access to the SAME ED API or will the paid apps have access to (far) more (and far more extensive) information ?
• Will the free app authors AND the paid app authors have access to the ED API at the same time? Or are the paid app authors given an early access to the API - so they can prepare their apps. In effect given them an advantage.

I can't recall having seen information about this by FD. This is something I really would want to know.
Perhaps someone from FD can comment on this.

Those questions have not been answered by FD but directly after that post Michael Brookes responded on the 26th of April 2015 with the following: "It is something we still want to support, but we do have finite development resources. Once we have some freed up I'm hoping we can push forwards with this. Michael". Source. Sadly in the same thread approx a month later on the 22th of May Michael Brookes wrote also: "It's still on our radar, but it is low on the current priority list. If anything changes we'll let you know. Michael". Source

It has been mentioned by commanders in that thread that all will be fine and we just have to be more patient (well the words were slightly less friendly). Tbh how much more patience do we need to have? I for one don't think we will see an external API any time soon - unless commercial apps will be created and those need an api, hence business case and money for FD.

That is why I'm asking for a "dump file" solution here and also in that thread in this post - were a was slightly less friendly and a bit harsh/disappointed. Please DO give tool authors something to work with.

In the mentioned thread Michael Brookes DID respond four times. It would be highly appreciated if he (or someone else from FD) could do the same here. Communication is key imho.

 

[Darth Ender]

I've seen games where third party apps are available and it never ends well with these kinds of games. You get privatized networks of players utilizing network intelligence that, while may be using the same api available to anyone, is not shared with everyone for the benefit of in-game activity. So no, this is not a good idea, it will do nothing but contribute to negative impacts on gameplay on a whole.

We already see what the impact is of crowd sourced applications without an actual api in this game - like the trade programs that are out. They ruin the trade route. They ruin the basic gameplay associated with discovery and exploration of such routes. They offer absolutely nothing but the devaluation of anything that is offered in the gameplay provided by FD.

These api's and programs on a whole, damage anything that elite dangerous has to offer for the sole purpose of making credits or undermining other players easier while not offering anything significant to improve the game unless you are one of the privileged with access to said programs or private networks utilizing such api's. This is regardless of how open source of freely available such things are made.

The only improvements to the game should be universal and from within the game. There should be no third party mechanism that gives players an advantage. Most certainly, FD should not make it easier for such a thing to exist on top of the means that such tools and players utilize already external to the main game.

The path of least resistance is the cancer of sand-boxy type games. FD should combat this at every possible angle to force the gameplay to be constructive and beneficial. Anything that min-max's your gameplay negatively impacts the game, anything that gives you a point of view outside of the single cmdr negatively impacts gameplay.

At least at this time.

 

[Agony_Aunt]

I would quite like an in-game best trade route calculator, as long as it was limited to the information available or obtainable by you in-game. Couple this with making market data tradeable in-game and you broaden the game and open up new career options.

That's decent enough.

 

[wolverine2710]

Going OT in my own thread, bad wolvie.
Brilliant news. Two commanders have taken OCR-ing one step further. Its now possible to OCR the ED system view. More info here. Also THE place to comment on them. Please don't make my mistake and go OT here ;->

• ExplOCR
• Elite Galaxy Online

Update: EDCodex has been out for exactly one month now. Celebration time!!!

 

[bootsam]

I prefer the community to do these kinds of things other than FD. It engenders involvement and promotes the community having a 'stake' in the game. Plus, with respect to FD, there are some clever bods out there who can come up with ideas better than they can as they can be more objective. Sometimes you cant see the wood for the trees. Besides they have a lot of work to do elsewhere.

FD could see their galaxy as the OS and the community can provide 'apps' or extensions for that OS (under approval and scrutiny of FD). They could sell this to other parties and before you know it we have a densely populated galaxy with all sorts going on.

I quite like to OCR trade data. I know theres an argument for keeping everything to yourself, but I prefer to share as that brings unforeseen things about. ie my quiet system might become more populated due to a profitable route, thus I get to say o7 to more and more peeps. I like seeing other cmdrs who share my more cooperative view of the galaxy.

I quite like what Inara is doing. Its this kind of community orientated service that I like. I have CMDR Mobius to thank for this as it was his group that took me from wandering in solo, about to pack it in and gave me a galaxy that I could share with others. I want to work with others, not against them. By working with them we can make the galaxy better rather than concentrating on combat. By going against the environment we can improve the environment. That has a benefit for everyone not just the few shooters. Making the environment more responsive, more intelligent with more variety.

Humanity has always worked better as a group rather than the individual.