Discussion Open Letter to Frontier Developments

[wolverine2710]

Hey,

It appears to me that there is really two parts to this discussion. I would encourage the authors of the letter in the future to be a little more clear in what they are specifically asking for if only to help focus the discussion of threads like this.

The two requests from this discussion that I see are as follows:

• General support from Frontier for third-party developers.
• Specifically support from Frontier for more easily getting market data from Elite to third-party developers.

The primary focus of the OP seems to be asking Frontier for general support but the discussion seems to focus on the market data. There are many forms general support from Frontier can come but the biggest is just getting data from the game to third-party developers. This data could be market data or not. It could also be information about power play, item stats, universe data, and much more. This of course if done amazingly right could be expanded to more such as chat and even inventory management.

As market data seems to be the focus of most the debate in this thread it is probably a good idea for Frontier to avoid that for a while if they were to officially support an external API of some kind. Launching an API is work enough, trying to also balance the design challenges is something that should come later once the basics are working.

If the discussion is really about requesting support from Frontier for third-party developers I would find it hard to do anything but agree and encourage that.

Before I go a bit further I feel it is important to clarify what third-party support can mean. Here is how I break it down myself:

• External APIs: These are generally HTTP APIs that allow third-party developers get information about the game out, and even sometimes into the game, via external applications.
• In-Client APIs: These are more like mods. You see a lot of it in WoW where you can download scripts that change the way your UI looks, adding features, and more.
• Static data: This is generally data that doesn't change between patches or expansions packaged together and given out. Really awesome for mobile apps.

I don't know how I feel about in-client APIs. It tends to change the game to much for my liking and generally leads to situations where you have to have certain things installed or others are just going to be better than you.

External APIs though offer a LOT of potential. Here are some examples of what external APIs can do for a game when the company supports them:

Game: Destiny
API: Inventory management
Access: Read and write
URL: https://chrome.google.com/webstore/...anager/apghicjnekejhfancbkahkhdckhdagna?hl=en
Description: Because Bungie has exposed a read and write API for Destiny third-party developers are able to create applications that allow players to adjust their inventory from outside the client. This can be done in real time. I don't have a console myself so am going off of what my friends who do play Destiny have told me but from what I understand this works while you are playing the game as well. So you can have the game open, adjust your inventory from your laptop or phone, and the change applies right away to your character.

Game: EVE Online
API: Fitting management
Access: Read and write
URL: https://community.eveonline.com/new...e-from-kill-boards-and-external-applications/
Bonus URL: http://cdn1.eveonline.com/www/newssystem/media/68768/1/fitting_import_demo.webm
Description: A really important aspect of EVE Online is saving and managing large numbers of what they call fittings. A saved collection of items that should go on specific ships. For a long time third-party developers have offered tools for EVE players that allow you to play around with fits from outside the client without having to actually have the ship or items. Just to sort of see what it does to the stats for a ship. With the addition of this API it made it incredibly easy for players to get fits from outside the client, often from one of those fitting tools, into the EVE client in real time.

Game: EVE Online
API: Universe data, corporation data, and more
Access: Read only
URL: http://evemaps.dotlan.net/
Description: Dotlan is considered a very popular tool for EVE players. Dotlan takes the same information you can get in the client and just presents it in a different way that players often prefer when planing. The format used by Dotlan often makes it easier to try and plan invasions and track how they are going. Dotlan does more though in that they try and track the history of things. So you can see the history of who owns a solar system, how many members a corporation (the EVE version of a guild) has and has had over time in nice graphs. There is a lot more but that should give you a general idea.

Game: League of Legends
API: Not sure of the exact APIs used
Access: Read only
URL: http://champion.builder.gg/
Description: By pulling stats of champions sites are able to build really awesome graphs showing how a champion does in different areas of the game over times. Often including things such as popular item set choices. Admittedly I don't know LoL well enough to pull up better examples of API usage and there are a lot so I just sort of picked one.

I could go on listing examples for a long time but the point I am trying to make is that there is a lot that could be offered to third-party developers without being a game design challenge. With that in mind I think the discussion in the thread could be a little different. There are a number of posts that seem to be against any kind of API because they immediately link it to market data and think that is the only thing that would be offered. Which again if I understand the OP correctly is not what they are asking for. I doubt they would complain if the market data was included but is not the specific goal of the letter.

I don't know how I feel about the idea of market data being more accurate or readily available as I never spent enough time hauling to really understand that. I do however fully support the request from third-party developers in asking Frontier to officially support them in some fashion with some data. There are so many things that could be offered and would make this game so much more amazing. True a lot of it could be built into the game but Frontier doesn't have an unlimited amount of money and I actually like having something to do related to the game I play while not at my computer.

Anyways bottom line is I whole heartily support this request!

One word: BRILLIANT.
A very good and informative post. It will have taken you quite a bit of time to create it!! Don't have time atm to go into it but its so much appreciated. The kind of post(s) I'm looking forward to see more in this thread. Well done commander, I salute you.

 

[Zac Antonaci]

Dear Tool Developers and clever people,

Thank you for your open letter. Reading through the letter there appears to be 2 main questions that I can pull from this.

The first is the down time of the API and question as to whether it will return due to it's downtime. I understand very much that this will cause concern.

With this specific concern the best bet (when it was down) is to report it on the bug forum and they will be able to confirm whether this is a bug or not. Alternatively you can the community team who would be happy to find out for you. (community@frontier.co,uk)

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

Other than that, thank you for your support with the community. Community created content and tools are a fantastic way to add more help support or richness to the game and we do appreciate your efforts greatly.

Yours sincerely,
Zac

 

[wolverine2710]

Originally Posted by Kristov I don't use any 3rd party tools, I WANT some way to know where I've been, something the game should actually be telling me in the first place, and if someone has a working 3rd party tool that does that without me having to manually input the information, great! Getting commodity prices from all over the bubble is quite a bit different from keeping track of where I'VE been personally, and if you can't see that difference...well...ain't no making you understand it, so I won't bother.

If FD wanted you to know where you had been obviously they would have put it in game. Your use of screenshots to keep track is immersion breaking and really tantamount to cheating. I mean your enthusiasm for an automatic export of your travels to a third party tool, without the need for manually and tediously inputting the data is obviously hacking. But then again I wonder why I bother reasoning with the inherently unreasonable.

A repost of this morning
Commanders. We are all passionate about ED, otherwise we would not play the game and be here in the forums. We all have a right to voice our opinion - pro or against the open letter. As I've signed the open letter, initiated EDDN and I'm the co-author of EDCOdex (website with a LARGE collection of tools,threads,videos) we all know where I stand.

A plea to you all. Can we please act in a civil manner and discuss the open letter in such a way that this thread is meaningful. The tone makes the music. I'm extremely passionate about getting an API, a real web-api, a dump api or official approval or disapproval of using the companion api. When we try to respect the opinion of each other the changes of this important thread getting derailed gets minimized and perhaps FD listens and we get an official statement from them.

 

[EUS]

The best, and long-lasting games are those that encourage the fans to create add-ons, etc.
The authors of the tools have my gratitude and full support.
They do it for the game, not money, which makes their work that much better.

 

[wolverine2710]

Dear Tool Developers and clever people,

Thank you for your open letter. Reading through the letter there appears to be 2 main questions that I can pull from this.

The first is the down time of the API and question as to whether it will return due to it's downtime. I understand very much that this will cause concern.

With this specific concern the best bet (when it was down) is to report it on the bug forum and they will be able to confirm whether this is a bug or not. Alternatively you can the community team who would be happy to find out for you. (community@frontier.co,uk)

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

Other than that, thank you for your support with the community. Community created content and tools are a fantastic way to add more help support or richness to the game and we do appreciate your efforts greatly.

Yours sincerely,
Zac

Can the devs please have a look at the suggested dump api. It really does not take much effort. A copy/paste from an earlier post addressed to someone else.

I understand your concerns but I don't think supplying us with an API has to be expense for FD - at least NOT a stop gap solution. A copy/paste from post #105. Since last year multiple authors have created threads were we basically stated: We know creating a good official API takes time/resources increases cpu usage and bandwidth on the FD servers BUT we have a suggestion. A stop gap solution. One which creates basically NO extra cpu usage on the FD servers, does NOT increase bandwidth usage and is really quite easy to implement. My words as someone who makes a living as a Java software engineer. Basically dump the information seen on screen when opening it in an XML/JSON file and let the third party tools collect it and distribute it. For more and detailed information see the thread "Open letter to FD - Please DO actively support third-party tools".

A disadvantage of that dump API ifs ofc that the data is available only on the machine where you run ED. That data can ofc be distributed by for example EDDN to websites and then used. But what if we want to display data on a smartphone? To keep things simple as an example, what if a tool author wants to create an android app which shows the same information as the official iPhone app. Tool authors have already thought about that. Gen.Fussypants has created two programs, an Android app "AndDiscovered - Android exploration tracking tool modeled after EDDiscovery" and a method to send data from a PC to that Android smartphone "EDProxy - Netlog proxy server" . The prosxy is oipen source and iirc the author is open to extend it for other uses. For example sending data to an android smartphone received from the dump API. Iḿ pretty sure it can be extended to gather all smartphones. Just my 2 euro cent.

Bottom line: The eco sytem to distribute is online since approx a year (EDDN), lots of sites use that distributed data, there are methods to send data directlly to (a) smartphone(s) . All we need is data from FD. Data which is sanctioned by them. Its amazing what have been done so far by scraping the (debugging) netlogs, OCR-ing and the companion API. Imagine what can be done if we have real data - say all data now available on the menus in the cockpit. The eco sytem would explode with new and brilliant tools. Its up to FD to decide what direction they want to go in the near and long ternm. Tool authors will adapt.

 

[CMDR Bloodhawk]

Dear Tool Developers and clever people,

Thank you for your open letter. Reading through the letter there appears to be 2 main questions that I can pull from this.

The first is the down time of the API and question as to whether it will return due to it's downtime. I understand very much that this will cause concern.

With this specific concern the best bet (when it was down) is to report it on the bug forum and they will be able to confirm whether this is a bug or not. Alternatively you can the community team who would be happy to find out for you. (community@frontier.co,uk)

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

Other than that, thank you for your support with the community. Community created content and tools are a fantastic way to add more help support or richness to the game and we do appreciate your efforts greatly.

Yours sincerely,
Zac

Zac, can you comment on why nearly a year has passed since we've been told there'd be a "few weeks" before more news on the API in this thread (https://forums.frontier.co.uk/showthread.php?t=99437&page=32)? It's hard not feeling mislead and in some ways deceived.

 

[bsnaylor]

A repost of this morning
Commanders. We are all passionate about ED, otherwise we would not play the game and be here in the forums. We all have a right to voice our opinion - pro or against the open letter. As I've signed the open letter, initiated EDDN and I'm the co-author of EDCOdex (website with a LARGE collection of tools,threads,videos) we all know where I stand.

A plea to you all. Can we please act in a civil manner and discuss the open letter in such a way that this thread is meaningful. The tone makes the music. I'm extremely passionate about getting an API, a real web-api, a dump api or official approval or disapproval of using the companion api. When we try to respect the opinion of each other the changes of this important thread getting derailed gets minimized and perhaps FD listens and we get an official statement from them.

100%...
+rep.
I'm not going to re-ignite issues regarding "immersion breaking". But simply say, if you have an issue with "immersion", you just don't touch these tools. My mate who is one of the top ED streamers refuses to use such, and I respect that fully and totally understand that way of playing, but there plenty of people who do want them, and love them, and the dedication these people put into creating such tools is a huge boost to the community, and they deserve our thanks and praise. o/

 

[wolverine2710]

Dear Tool Developers and clever people,

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dev team are aware of the desire and it has been considered.

For starters I know you are just the messenger and we all know we should not shoot the messenger. That said.

Wrt to your post, with all due respect, its along the line of what Michael Brookes and David Braben said earlier - this year, last year. Yes we like the idea, but no timeline. As in in no new information. This to me is not helpful. It could be interpreted in a lot of ways but its not giving me the feeling that something is really been done to support the third party tool authors. I apologize for being harsh but on the 15th of January 2015 Michael Brookes asked us for input, after that radio silence. 4+ months before that in 2014 he said (when the data access policy changed and EMDN - EDDN basically- was shutdown) something along the lines of "we would like to provide a web-api"

What really makes me shudder/cry is the "and it has been considered." part of your post- past tense !!! It really does not give me a good feeling and I put this extremely mildly. When I take the gloves off I want to say something completely different. I can't because I've repeatedly said here that we should all try to be civilized and respectful, my bad...

 

[Cmdr Thrudd]

What frustrates me the most is that there is already part of an API, the iPhone app endpoint already exists. Spending a couple of dev days copy pasting the existing code, adding a security layer (API key system) and a quick page or two of documentation and we'd have a good starting point. Other features could be added over time but it would be nice to at least get the ball rolling.

 

[DCLXVI your God]

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

Rather vague.

 

[golan]

Incorporating the tools in the game is a design decision - it's optional and while I would disagree with not having such information in-game, I can accept it and it has a thematic dimension for me, the way it is now, these tools add a frail feeling of increased certaintity.

But having these tools at least externally, is a major thing for a lot of players. I would COMPLETELY ditch trading without these tools, period. And if the system is overhauled and prices will be made to change organically, in time, without such tools, trading becomes quite unpredictable and unattractive. There would still be a core of players who will do better without them, because they like this stuff, but the rest of us are not like that and we see trading just a means to an end: upgrading that nice ship to use it for whatever purpose we want it to that is not as lucrative (exploration in my case). When it stops being as good / efficient as other ways, we'll choose the more profitable path.

I think it only makes sense to let the community help with this part of the game, instead of investing a lot of development effort to have it in-game and they seem willing to do it, anyway. Please Frontier, collaborate more with them

 

[IronGremlin]

This is a more complicated decision then just data dumping session information to any validated requestor.

How is api request traffic prioritized?
How is it routed?
Are the request responses cached?
Where are they cached?
How frequently is the cache refreshed?
How do we prevent the exposed interface from getting accidentally spammed into oblivion by a novice?
How do we prevent the interface from getting deliberately spammed into oblivion by malicious software?
How do we isolate the API interface so that if it is attacked it doesn't compromise core infrastructure?


The iPhone App interface might have some of these questions answered, but it's super possible and in fact, considering the responses, super likely that some or all of these questions aren't answered in a way that is properly future proofed.

It's fairly obvious that the current API is piggy-backing on top of existing player session architecture, which means that any changes to that architecture (read: Frontier, fix your instancing mechanic s already! Posts) would likely also necessitate changes to the API.

Hence, committing to continuing to support an API that is used for it's originally designed purpose by basically nobody is a non-trivial expenditure of resources.

I'm not, at all, arguing against the importance of Frontier continuing to support the existing API, or the possibility of eventually extending it.

I'm just trying to point out that it's not a simple as just "data dumping."
Just because the goal you want seems simple to achieve does not mean that it is simple to implement sustainable solution at scale on top of existing production architecture.

There are factors at work here that Frontier isn't going to (and probably shouldn't) share with us. Please try to respect them as professionals and take them at their word when they tell us that this isn't as simple as we make it out to be.

 

[Kristov]

A repost of this morning
Commanders. We are all passionate about ED, otherwise we would not play the game and be here in the forums. We all have a right to voice our opinion - pro or against the open letter. As I've signed the open letter, initiated EDDN and I'm the co-author of EDCOdex (website with a LARGE collection of tools,threads,videos) we all know where I stand.

A plea to you all. Can we please act in a civil manner and discuss the open letter in such a way that this thread is meaningful. The tone makes the music. I'm extremely passionate about getting an API, a real web-api, a dump api or official approval or disapproval of using the companion api. When we try to respect the opinion of each other the changes of this important thread getting derailed gets minimized and perhaps FD listens and we get an official statement from them.

My personal feelings are what they are, and I still support this initiative fully despite my personal feelings on the matter as I believe that what the people making these tools are doing is worthwhile and should be supported as it fosters the community of Elite Dangerous in many different ways, all for the better.

Zac's response indicates to me, combined with Michael and David's statements on the subject, that FD is aware of the desire, but there's no plans to do anything about that desire. Personally I believe it's due to security concerns on FD's end, something I understand as I'm in IT and security is always a big deal factor in anything concerning access. The Apple app they put out, which appears to do nothing but show where you are docked and that's it, seems to indicate that FD isn't really keen on sharing database access even with their own app. That doesn't give much hope for what's being requested does it...

 

[AnthorNet]

Dear Tool Developers and clever people,

Thank you for your open letter. Reading through the letter there appears to be 2 main questions that I can pull from this.

The first is the down time of the API and question as to whether it will return due to it's downtime. I understand very much that this will cause concern.

With this specific concern the best bet (when it was down) is to report it on the bug forum and they will be able to confirm whether this is a bug or not. Alternatively you can the community team who would be happy to find out for you. (community@frontier.co,uk)

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

Other than that, thank you for your support with the community. Community created content and tools are a fantastic way to add more help support or richness to the game and we do appreciate your efforts greatly.

Yours sincerely,
Zac

Dear Zac,
Thanks for your answer, it's a start!

We know that holidays can be a busy time (or not at all!), but sincerely I think that this statement is something we already heard multiple times!
I really hope, you or someone else will get back to us with some other things to say, and we deserve it.

You can't always respond with some vaguely statement about something that get so less time to implement as other said.
We're mostly all professional software developers here, so we have a clear statement on how much time some things could take to be implemented.

Some very quick solutions were proposed that would not take time or resources from FD to be implemented.
Specially when all the structures needed by third party tool already exists and can be extended on our side to spread the data.

All things said, it's pretty easy to contact us by email if you prefer to engage in a more constructive dialog in private.

 

[Monk]

My personal feelings are what they are, and I still support this initiative fully despite my personal feelings on the matter as I believe that what the people making these tools are doing is worthwhile and should be supported as it fosters the community of Elite Dangerous in many different ways, all for the better.

Zac's response indicates to me, combined with Michael and David's statements on the subject, that FD is aware of the desire, but there's no plans to do anything about that desire. Personally I believe it's due to security concerns on FD's end, something I understand as I'm in IT and security is always a big deal factor in anything concerning access. The Apple app they put out, which appears to do nothing but show where you are docked and that's it, seems to indicate that FD isn't really keen on sharing database access even with their own app. That doesn't give much hope for what's being requested does it...

I don't think FD should give full access to the database. That would be too much power given to the players, imo. I think the data should still be driven by player activity which allows for some of the far and away places to have their secret runs.

Rather, the current API and method of access should be maintained but with the following additions/enhancements:

• Optimization to reduce load on the ED Database and to handle more messages with more data.
• A thin SLA. Nothing that punishes FD but a simple commitment to do their best to keep the API running after updates and when it goes down.
• Publicize the API so developers can take full advantage of all its functionality
• A few more datapoints to go along with the data that is shared today
  • Star coordinates
  • Station Distance
  • More Station information (economy, state etc)
  • Powerplay stats of system
  • Recent commodity price history
  • Traffic

 

[golan]

There are multiple reasons why they might want those APIs not working. They've just added new resources, new missions and re-balanced the game, so I imagine they might be interested to see the real impact of these changes unaltered by external tools. We don't know anything for sure and it would be nice to have more info from the developers and I do hope they choose to work more closely with that part of the community ...

 

[themroc]

The second and deeper question is the one about an API. We understand that there are lots of people who would like to have this API and it is something we have discussed. However, being able to balance when it will be possible is another question.

I cant give you a date or a rough time but I can at least assure you that the dec team are aware of the desire and it has been considered.

I'm the creator of EDDB and participiated in the creation of the Open Letter.

I'm not sure if you understand what our main goal is. A pretty and powerful API of course is one thing, which we all want somehow. Way more concerning though is the fact, that every little piece of data we obtain is neither fish nor fowl.

The Companion App API is widely being used for our purposes. Unfortunately its use brings alot of issues:

• We never received an OK from Frontier Developments. So basically we all use it in a twilight zone scenario. For me this is the most important part here. Please tell us your position on this!
• The outfitting and shipyard information is sometimes very wrong. Have a look at this EDDN error log. Very often a station doesn't have a shipyard or outfitting facility but still receives data of ships or modules. I can filter out some but not all of them. This makes a very bad reliability.
• Especially module information is often wrong and lists modules that are actually not being sold.

Next thing: Powerplay data. A part of the community takes it very serious and those people depend on stats on the matter. Tools can provide this and fortunately you started to give us data in this forum thread! And again - it doesnt work:

• If the data is not at least in a cycle up2date, it becomes useless. The last dump is 2 weeks old. Every tool that depends on it is broken now.

So maybe it's a start to fix that in the first place?

But most important: Please speak with us!

A simple conversation about what's possible and what not would be a simple yet very powerful process. Also, I'm sure nobody expects you to give us that powerful API tomorrow. We would be soooo happy already if you give us anything. We can be happy with the most simple data. A single thing can already bring joy to many players if manifested in 3rd party tool.

Please give it a try.

themroc

 

[moosehead]

I am surprised that Slopey's is not included!

 

[Darian]

What a load of rubbish!

No ship/Lorry loads a cargo without knowing where it is going and for what price it gets for carrying it. Just check out this place, https://en.wikipedia.org/wiki/Baltic_Exchange, since 1774 it's been supply shipowners with cargo and trade info and yet in the 33rd century you cannot have something like it!

sorry i forgot to tag it irony

 

[bigtaff]

sorry i forgot to tag it irony

NP Shipmate!