Discussion Existing API resources
- Thread starter nrage
- Start date
Having just gotten some code working to use the API I wrote some documentation. The main weirdness is the use of HTTP response codes in the form of redirects to determine if you need to login, and if so if you've succeeded at each stage.
NB: I've been updating those docs slightly since first posting here. Take note of the bits about 'discard' cookie and having to handle redirects yourself (else parsing HTML to check what page you're on).
Last edited:
Thanks! This is an excellent starting point.
I wrote a primitive scraper in Ruby many months back, which worked well enough for me to stick it in a cron job. https://gist.github.com/nickludlam/888aae12b126cd904fb1
Last edited:
Time to become more proficient in python it seems
Or you could puzzle over my perl: https://github.com/Athanasius/fd-apiTime to become more proficient in python it seems
Looking at Athan's Perl code I found that you need to supply an user agent compatible with Apple's Safari for iOS, otherwise the API returns 403 Forbiden. Tests I done using wget on a Linux shell were returning 403s both with wgets default UA (whatever that is) and when supplying Firefoxes UA (Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0) as a a UA.
The one that worked was the one from Athans script: "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12B411"
Are you sure that's Python?
Doesn't look like any Python I've written (and Captain's Log is written in Python)
In fact I'd say that's Ruby
Regards o7
Time to become more proficient in python it seems
Oops, I'm being horribly derpy and it's actually written in Ruby! I corrected my post above. In any case, if you want some help debugging then I'm sure some of us who are likely to frequent this board can help.
Credit where it's due, I got that out of https://bitbucket.org/orphu/edapi/src/ae95e51eaccb7e3faab4b670d0df1d830ad99313/edapi.py?at=master
There is a lot of stuff on the EDCodex, maintained by Wolverine. It was the only place we could share API code before the approval of use.
EDIT: This will take you to the EDDN stuff: http://edcodex.info/?m=search&cat=9
EDIT: This will take you to the EDDN stuff: http://edcodex.info/?m=search&cat=9
Last edited:
wolverine2710
Tutorial & Guide Writer
And the little read devil takes revenge for me mispelling his name 2-3 times ;-> Typos I swear.... Its ofc wolverine2710, not wolverine ;-) If only I had taken that name of the FD forum when it was still available. Seriously, thanks for pointing him and I hope others to EDCodex. He might just find more info/entries then he bargained for, grinnn...
Wolvie signing off.
Last edited:
Eureka!
Eugh! Finally! I now have some C# code which can authenticate with the API servers and download a profile
The biggest hurdle was that the .NET library really does not like the cookies returned by the API server, specifically the comma in the expires= value. In addition to this, it has a mind of it's own when it comes to whether it will send a cookie, or not - likely something to do with domain, path, or expires. In the end I had to code up a custom cookie parsing routine, and then make sure that I handed it a cookie devoid of all the options containing just the cookie name=value to send out.
So, I have to appologize to Frontier for making something like ~50 authentication attempts over the last couple of days, it won't happen again. On a related topic I have only downloaded the profile once, and my code will ensure I never download it more than once every 5 minutes at most (as mentioned in the other thread), so hopefully that will appease you?
I must say, WOW, the profile contains a lot of interesting stuff.. now to decide what I can usefully do with it.
I plan to share the code once I have completed the JSON parsing step which is next on my TODO list. Might even go so far as to include some netlog scraping code, we'll see.
Eugh! Finally! I now have some C# code which can authenticate with the API servers and download a profile
The biggest hurdle was that the .NET library really does not like the cookies returned by the API server, specifically the comma in the expires= value. In addition to this, it has a mind of it's own when it comes to whether it will send a cookie, or not - likely something to do with domain, path, or expires. In the end I had to code up a custom cookie parsing routine, and then make sure that I handed it a cookie devoid of all the options containing just the cookie name=value to send out.
So, I have to appologize to Frontier for making something like ~50 authentication attempts
over the last couple of days, it won't happen again. On a related topic I have only downloaded the profile once, and my code will ensure I never download it more than once every 5 minutes at most (as mentioned in the other thread), so hopefully that will appease you?I must say, WOW, the profile contains a lot of interesting stuff.. now to decide what I can usefully do with it.
I plan to share the code once I have completed the JSON parsing step which is next on my TODO list. Might even go so far as to include some netlog scraping code, we'll see.
I wonder if someone at Microsoft mis-read RFC6265 - 5.1.1 Dates and is simply barfing on the Day of Week. I can't blame 'em as there's no mention of day-of-week there, but I do note that RFC6265 - 3.1 Examples does contain "Set-Cookie: lang=en-US; Expires=Wed, 09 Jun 2021 10:18:14 GMT" as an example. To be fair to that RFC 'Dates' excerpt it does seem defined as to allow any amount of extra text in a date, just so long as it can find all of "day-of-month, month, year, and time".Eugh! Finally! I now have some C# code which can authenticate with the API servers and download a profile
The biggest hurdle was that the .NET library really does not like the cookies returned by the API server, specifically the comma in the expires= value.
If you're positive this it is .NET's behaviour to barf on that I'd file a bug report with Microsoft.
Last edited:
IIRC from my various googlings it seems like this is a "well known" issue in the 4.0 runtime which was "fixed" in 4.5. I only have Visual Studio 2010 (runtime 4.0) at home but I did give it a quick try using 4.5 at work, however as I was experiencing the "failure to send cookies back" issue at the time I did not specifically notice if the other issue was in fact resolved.
Last edited:
I was thinking: there seem to be a few of us who are writing the same basic code. Would it make sense to turn this in to a dll and share it? We can then fix all the little gotchas that we find (the whole cookies thing and FD redirecting all over the place is a little annoying).
And on a similar basis, once we have the required cookies it would be nice to put them in a more sanely accessible place. I'm currently writing them to a JSON file in %APPDATA% to make them more easily available to other apps in some possible future.
And on a similar basis, once we have the required cookies it would be nice to put them in a more sanely accessible place. I'm currently writing them to a JSON file in %APPDATA% to make them more easily available to other apps in some possible future.