Compressed binary stream with fixed-width string fields or eliminate the string fields entirely and use GUIDs.
It has to run on an end-user system while they are also running the game.
Last edited:
Discussion External API Requirements Thread
- Thread starter Michael Brookes
- Start date
- Status
Compressed binary stream with fixed-width string fields or eliminate the string fields entirely and use GUIDs.
It has to run on an end-user system while they are also running the game.
Michael,
Can we clarify if you mean an API hooked into the client, or an API on the webserver?
[...]
Although I don't think coordinates are included in that, there's lots of guys that would current system coords handy
I overlooked this thread somehow, and have not read it completely, but I think API data will (and should) be limited to anything we already have at the client, so actual system name, commodity prices & availability and stuff like this, and from the System Map only the known part, if it is not explored completely. From the Galaxy Map it sure would be nice to get the real coordinates ready packed. The actual GM does only show the direction & distance to other systems, and if you do not reveal the coordinates, it would be nice to have at least these distances to other systems via API. Would be sufficient to alculate the coordinates by my own or let EDSC do this. Right now searching some reference systems, typing the names, switching to FF & typing the distances by hand is really annoying.
I would certainly like a personal history type of app...where I've been, what market prices were like, outfitting choices, etc.
Included with that I would like an option to save system information, # of stations, types, slots available, etc. PLUS the current faction standings within any system, updated if I am not able to visit. This 'small bit of live information' would be nice, it would keep me involved within the game,and is generally available within the Galaxy Map.
Included with that I would like an option to save system information, # of stations, types, slots available, etc. PLUS the current faction standings within any system, updated if I am not able to visit. This 'small bit of live information' would be nice, it would keep me involved within the game,and is generally available within the Galaxy Map.
Possibly (with an ingame http server answering JSON or XML queries f.ex.) there may be another computer running at home for collecting/distributing those data.
The client end is the least of the problem: ED is really pretty light on CPU. I suggest you actually establish that there is a performance issue before you start optimising for it.
A binary format is a bad idea unless there is some particularly good reason for it. I don't see any reason for it here. See http://www.faqs.org/docs/artu/ch05s01.html for a short discussion on why binary formats are unwise.
I don't really disagree. It was a slightly flippant response to your use of SLAs as the primary justification for a message based API. My point is that if FD can't keep a query API up (and responding quickly) then the 3rd party API is the least of our problems because the game itself uses query APIs extensively (or at least appears to). Also the value of a 3rd party app is almost entirely tied to the availability of the game so you're already in the situation you fear. So I don't see potential SLA issues as a significant factor in this case. Note that downtime for a message API is possibly harder to recover from (for FD) than downtime for a query API: if FD doesn't have a log of the messages that should have sent out while the service was down then those updates are lost and all clients will have some bad data until that data is next changed. Also caching is equally applicable to a query API as well. Finally a poorly designed message API can be quite inefficient if it is pushing messages that a client isn't interested in.
Personally I would prefer a message API because there are applications where a push model is much more efficient (something which tracks market prices over time, for example), but I'm not that bothered. Practically I think the API must have a query capability whether or not it also has messaging. If I need the market prices for system X I don't want to have to wait until they next update to get them.
I don't think that any information should be available that is not right there on the game's screen - current market prices, current system, nearby systems and distances, galaxy map grid coordinates, etc. I don't want to see mass scraping of the market data for hundreds of different systems.
Suggestions slowing down, I guess the main requirements have been exposed
Like others have said, while waiting for a secure full-fledged API, having rapidly the possibility for the game client to dump to log the info available from the web-api + star coordinates would go a long way, and would probably not be much effort.
Like others have said, while waiting for a secure full-fledged API, having rapidly the possibility for the game client to dump to log the info available from the web-api + star coordinates would go a long way, and would probably not be much effort.
Some basic rules
1. There should be some principles that guide what people can get from the API.
2. As a strategy, I suggest the API should make available everything that the user can access via the standard interfaces. The subset that users will get with v1.0 of the API should be a matter of prioritising what's possible (the purpose of this thread!).
3. Could an API server reside on a http port of the ED client app? This would enforce 1(a), and reduce web traffic for Frontier Dev, but might have security implications on a LAN (requiring some authentication, etc).
1. There should be some principles that guide what people can get from the API.
- Only data that is currently available in a player's standard interface, should be available in the APIs. i.e. APIs do not reveal undiscovered secrets.
- It follows that API responses are user-centric, and may only be used to access the user's own account. Perhaps also require that the user is logged into the game.
- Allowing actions (equivalent to POST/PUT, etc) may encourages over-automation (bots). Ideally, achieve the balance of helping players with light automation, rather than allowing a bot to do all the work for them.
- There are many advantages for only publishing current data; third parties (rather than the API) to be given the responsibility to accumulate and curate data. Otherwise, the official offering would be presenting inconsistent experiences to some users.
2. As a strategy, I suggest the API should make available everything that the user can access via the standard interfaces. The subset that users will get with v1.0 of the API should be a matter of prioritising what's possible (the purpose of this thread!).
3. Could an API server reside on a http port of the ED client app? This would enforce 1(a), and reduce web traffic for Frontier Dev, but might have security implications on a LAN (requiring some authentication, etc).
I think API for market prices, station information, etc. should be web-based, otherwise it's just pain for web-based tools. Client-side API for this just lowers user comfort (in case that player must upload his log to a site) or increases dependency on other tools/utilities (which will get data from client and sends it to the website) and it definitely increases chance of errors and mistakes (in both cases).
But I can personally imagine something like it was mentioned, for first API versions - data can be retrieved just for current online user, for current station where is he docked, etc.. It just means that he will press "refresh data" button on the tool website while he is in the game and the tool will get his actual data (including station, star system, whatever).
I was referring to the standard 'REST' API, which runs over HTTP. They're not exactly web pages, but you can view the raw responses (to a well-formed request) using a web browser.
Indeed, I see the benefits of an ED-server-provided API (off-LAN apps, and so on).
I see the ideal architecture for third-party apps as being:
* The ED server, which serves requests from
* The third-party server, which serves requests from
* The player's client.
How does this help? Well, a server will be better-equipped than a phone or tablet, to compute a pathfinding route across 100 ly, or to store all the data you'll have accumulated on your travels. The third-party server could introduce the idea of off-game alliances, where data may be shared for mutual benefit. There's more, but I think this thread is long enough without my help...
I agree. Market prices was a bad choice of example as you should only have access to the current station's prices. There is plenty of data you can get on other systems though, e.g. faction info, trade info, etc.
This is why I suggested an Atom feed as the messaging API, which is easier to recover from (just page back to where you last were). I'm not in any way suggesting a push model. Query APIs and push models are the wrong way to go when Atom feeds and pull are available.
But all of this really depends on what kind of API we are talking about anyway. Has it been resolved whether we are talking about a server API or a client API? I don't think that question has been answered yet, and so before we know that, it's really hard to make any recommendations.
So fellow hacker Iain M Norman and I were guests on Kerrash's Twitch Stream last Friday, and we were having a community discussion on what kind of apps people might find useful.
Aside from the people looking to make their own peripherals or data displays which augment the existing client experience (which I personally think falls within the remit of client "addons" rather than an external API), an overwhelming number of people were thinking of an activity tracker / pilot's log application.
The idea of the application I have in mind is that it starts out as a private repository of information, where you can refer back to where you've been and what you've done. Ultimately I'd love it to become as big as DICE's Battlelog webapp which for me is one of the best game social tools I've used. It's well designed, comprehensive and real-time with some very smart streaming tech at the back-end.
Second order stats
The next step is to provide some second order functionality by analysing the data we've collected, and giving you some insight into where the best trade routes are, when you had your biggest kill streak, best profit per hour, most cost-effective weapon, the last time you went exploring. It's also
The next step from that is to layer on top a social graph where you can group together and see friend's stats, coordinate efforts and compete with each other in as much of a playful or serious way as you want.
Third order stats and social tools
Finally, once you have stats and friends, you can start work on higher level functionality, and these are some of the ideas generated:
- Player-generated missions: Declare a period of time and the rules, and invite others to participate in some externally verifiable activity. ie/ "Manic Miner", who can mine the most valuable or greatest volume of ore in a given period. Include and exclude particular types of mineral.
- Player groups which allow every member to view the activity of each member, and understand how cumulatively you're affecting the galaxy.
- Multi-hop racing between systems where we verify your route and the timings via an API
- "Top trump Commanders" using your friends list and assigning metrics to a set of stats according to the real data
Architecture
I'm not clear as to what the appropriate architecture for this would be. It can either:
- Commanders install a client app which hits the Frontier API, it pulls data back and then posts it to a central website via another API
- Commanders create an account on a website and authenticate (OAuth2?) with Frontier to give the website some keys to poll the API on their behalf. This is then done periodically, or some sort of stream is set up to receive events as they happen, as Twitter's streaming API does.
API data
This is a fairly exhaustive list of the activities we'd love to see made available via an API call. Personally I'm a big fan of JSON data obtained via REST/HTTPS calls:
Systems you've jumped to
Stations you've docked at
Chat log
Scan log (NPCs and other human players)
Trade log (including black market)
Kill log
Deaths
Missions log (completed, failed, branched?)
Bounties and bonds log
Explorer discoveries log
Explorer sales log
Ships and module transactions
Faction rep changes
Naval rank progression
Permits obtained
Trade/Combat/Explorer ranks obtained
Time spent fuel scooping (or tons scooped)
Times you were interdicted
Shot accuracy
Damage dealt (and with what weapon)
Damage taken (hull and shields)
Travel log
- Supercruise
- Stations
- Nav beacons
- USS & RES
Items scooped (cargo and ore)
All of the items marked 'log' would need some sort of timestamp associated with the data to allow reconciliation in the back-end database. There was also a question over whether Frontier could supply a Friend list of the commander hitting the API, allowing the pilots log website querying the API to automatically suggest friends who've also signed up to the pilots log website.
We also had some questions about the nature of the API and app development:
- For efficiency, what is Frontier's thoughts around API usage rate limiting, and how frequently we can ask for real-time pieces of information like a player's location?
- Can you charge for a mobile app that utilised a web service which was collecting data from Frontier's API?
Thanks!
Nick / Commander Bewilderbeest
Aside from the people looking to make their own peripherals or data displays which augment the existing client experience (which I personally think falls within the remit of client "addons" rather than an external API), an overwhelming number of people were thinking of an activity tracker / pilot's log application.
The idea of the application I have in mind is that it starts out as a private repository of information, where you can refer back to where you've been and what you've done. Ultimately I'd love it to become as big as DICE's Battlelog webapp which for me is one of the best game social tools I've used. It's well designed, comprehensive and real-time with some very smart streaming tech at the back-end.
Second order stats
The next step is to provide some second order functionality by analysing the data we've collected, and giving you some insight into where the best trade routes are, when you had your biggest kill streak, best profit per hour, most cost-effective weapon, the last time you went exploring. It's also
The next step from that is to layer on top a social graph where you can group together and see friend's stats, coordinate efforts and compete with each other in as much of a playful or serious way as you want.
Third order stats and social tools
Finally, once you have stats and friends, you can start work on higher level functionality, and these are some of the ideas generated:
- Player-generated missions: Declare a period of time and the rules, and invite others to participate in some externally verifiable activity. ie/ "Manic Miner", who can mine the most valuable or greatest volume of ore in a given period. Include and exclude particular types of mineral.
- Player groups which allow every member to view the activity of each member, and understand how cumulatively you're affecting the galaxy.
- Multi-hop racing between systems where we verify your route and the timings via an API
- "Top trump Commanders" using your friends list and assigning metrics to a set of stats according to the real data
Architecture
I'm not clear as to what the appropriate architecture for this would be. It can either:
- Commanders install a client app which hits the Frontier API, it pulls data back and then posts it to a central website via another API
- Commanders create an account on a website and authenticate (OAuth2?) with Frontier to give the website some keys to poll the API on their behalf. This is then done periodically, or some sort of stream is set up to receive events as they happen, as Twitter's streaming API does.
API data
This is a fairly exhaustive list of the activities we'd love to see made available via an API call. Personally I'm a big fan of JSON data obtained via REST/HTTPS calls:
Systems you've jumped to
Stations you've docked at
Chat log
Scan log (NPCs and other human players)
Trade log (including black market)
Kill log
Deaths
Missions log (completed, failed, branched?)
Bounties and bonds log
Explorer discoveries log
Explorer sales log
Ships and module transactions
Faction rep changes
Naval rank progression
Permits obtained
Trade/Combat/Explorer ranks obtained
Time spent fuel scooping (or tons scooped)
Times you were interdicted
Shot accuracy
Damage dealt (and with what weapon)
Damage taken (hull and shields)
Travel log
- Supercruise
- Stations
- Nav beacons
- USS & RES
Items scooped (cargo and ore)
All of the items marked 'log' would need some sort of timestamp associated with the data to allow reconciliation in the back-end database. There was also a question over whether Frontier could supply a Friend list of the commander hitting the API, allowing the pilots log website querying the API to automatically suggest friends who've also signed up to the pilots log website.
We also had some questions about the nature of the API and app development:
- For efficiency, what is Frontier's thoughts around API usage rate limiting, and how frequently we can ask for real-time pieces of information like a player's location?
- Can you charge for a mobile app that utilised a web service which was collecting data from Frontier's API?
Thanks!
Nick / Commander Bewilderbeest
Last edited:
something like Cmdr´s Log here https://forums.frontier.co.uk/showthread.php?t=96350 would be great to have ingame
Locked target data
I'm not sure if anyone has said this, but I would love for the API to include the tools so an app could be developed to display formatted information on the locked target including scan info. I play with trackIR a full flight setup. This includes a mounted car phone holder on my desk that is opposite of my second monitor. I usually use this for flight sims to either show telemetry data or extra buttons. It would be spectacular if the API included the ability to export data like that to display on another device. Just imagine the rigs people might build if they could incorporate tablet touch screens with telemetry and data into their cockpits.
I'm not sure if anyone has said this, but I would love for the API to include the tools so an app could be developed to display formatted information on the locked target including scan info. I play with trackIR a full flight setup. This includes a mounted car phone holder on my desk that is opposite of my second monitor. I usually use this for flight sims to either show telemetry data or extra buttons. It would be spectacular if the API included the ability to export data like that to display on another device. Just imagine the rigs people might build if they could incorporate tablet touch screens with telemetry and data into their cockpits.
For a while, a few years I guess, I have been enjoying tinkering with php and mysqli, and I know there are better ways to code, but I have been enjoying playing Elite Dangerous and decided to make my own database so that I could see what goods were in the stations I regularly travel to/from to trade.
One of the many key elements, is trawling the files at C:\Program Files (x86)\Frontier\EDLaunch\Products\FORC-FDEV-D-1002\Logs to glean the current solarsystem name, such as {08:20:40} System:6(HIP 75039) Body:29 Pos-12.6058,599.265,-800.452) .
Now after todays update, 22/01/2015, this information no longer appears in the file!
Please Frontier, what have we dontr to upset you????
I know your looking at releasing an api, but as you have not yet released it, could we please get this info back ?
One of the many key elements, is trawling the files at C:\Program Files (x86)\Frontier\EDLaunch\Products\FORC-FDEV-D-1002\Logs to glean the current solarsystem name, such as {08:20:40} System:6(HIP 75039) Body:29 Pos
-12.6058,599.265,-800.452) .Now after todays update, 22/01/2015, this information no longer appears in the file!
Please Frontier, what have we dontr to upset you????
I know your looking at releasing an api, but as you have not yet released it, could we please get this info back ?
Ship operations really...
1. Some way for you to gain telemetry about your ship (landing gear/cargo scoop state, speed, super cruise state, current target, etc, etc).
This way external applications can be written that integrate with ED and so can be used to augment current functionality (I'm think of voice control like Voice Attack but cleverer as it would know the state of your ship and have features that can use it).
2. Some basic ways to query and search for specific UI panels and select options by name instead of only by keyboard presses etc.
1. Some way for you to gain telemetry about your ship (landing gear/cargo scoop state, speed, super cruise state, current target, etc, etc).
This way external applications can be written that integrate with ED and so can be used to augment current functionality (I'm think of voice control like Voice Attack but cleverer as it would know the state of your ship and have features that can use it).
2. Some basic ways to query and search for specific UI panels and select options by name instead of only by keyboard presses etc.
I have a couple of use cases that I'd like to be able to write apps for, both related:
1) Exploring Logging
* be able to build map of my explored systems
* be able to look at individual objects in systems
* be able to track where I was at any given point
* be able to add my own narrative/photoshots to interesting systems/objects
2) Buckyball Tracking
* be able to trace where a particular player is at this point in time
* be able to publish data privately, to groups, or publicly
* I'm envisaging a spectator app for the buckyball rally (or any similar event) - showing racers, times etc
Both are probably the same underlying data. But it would be nice to be implemented by installing a specific module in the ship to record and transmit - this could be configured by the player to the required privacy level.
API wise, ReST would be nice (I'm a java/python type of guy), but anything non language specific would work.
It might also be better if you never do this - playing the game *and* developing apps for it might cause the end of my marriage ;o)
1) Exploring Logging
* be able to build map of my explored systems
* be able to look at individual objects in systems
* be able to track where I was at any given point
* be able to add my own narrative/photoshots to interesting systems/objects
2) Buckyball Tracking
* be able to trace where a particular player is at this point in time
* be able to publish data privately, to groups, or publicly
* I'm envisaging a spectator app for the buckyball rally (or any similar event) - showing racers, times etc
Both are probably the same underlying data. But it would be nice to be implemented by installing a specific module in the ship to record and transmit - this could be configured by the player to the required privacy level.
API wise, ReST would be nice (I'm a java/python type of guy), but anything non language specific would work.
It might also be better if you never do this - playing the game *and* developing apps for it might cause the end of my marriage ;o)
- Status