Release Questions about a replacement for Maddavo's updating for TradeDangerous

Hopefully, most of you know Oliver (@kfsone) wonderful tool, TradeDangerous (TD), which is now being semi-maintained by @gazelle (Bernd) here: https://bitbucket.org/bgol/tradedangerous

The beauty of this tool is in its completeness and complexity. It is a route planner, a market database, a buy & sell location finder, and, most importantly, a complex trader calculating engine for multi-stop/multi-hop straight routes or loops. Until a few months ago, it was regularly updated by maddavo (http://www.davek.com.au/td/). Maddavo's market data (MMD) synced with EDDB daily, and was (semi-)manually updated by other tools (for example, EDMC can write station data in a format which can be updated to MMD). More importantly, MMD also had an EDDN listener. Therefore, MMD provided a full price list which was updated ebvery 24 hours, a rolline 48 hour last change list updated every 2 hours, and a rolling 3 hour new prices list updated regularly ("after a batch of data is processed"). There is a plugin for trade dangerous whoich can read these files, or they can be imported directly from the link on Maddavo's database as they are in the TD native prices" format. All was well in the world of the lite trader.

Unfortunately, over the past 4–6 months, maddavo's site has been almost completely static. Once in a whjile it updats for a day, and then remains quiet for weeks. My question to the coding experts and E:D lovers here is how difficult would it be to replicate/replace that functionality? Obviously this would have to be hosted somewhere too, but one step at a time.

Granted, that's probably asking a lot, just looking at the TD source code (including the plugins). As a hopefully more manageable question, how difficult would it be to be able to take a daily pricing download from EDDB or wherever, and munge it into a form which TD could import? I know that the listings.csv file on the API page of EDDB is not the native "prices" format; I've tried. Software that can be manually run to download data would be a big step! My minimal programing experience is in statistics (which is why I was able to upgrade the penalty function for distant stations); I know nothing about SQL or webscraping (and use R more than Python, although I do use it).

Any suggestions, or explanations as to why this is not a feasible task, would be greatly appreciated. Thank you.

CMDR Avi00013
 
hello CMDR Avi00013,

i would absolutely and perfectly appreciate this!!
(and i think there are others too)

(despite the much more important things around, currently ;) )

thanks,
nepo
 
Hopefully, most of you know Oliver (@kfsone) wonderful tool, TradeDangerous (TD), which is now being semi-maintained by @gazelle (Bernd) here: https://bitbucket.org/bgol/tradedangerous

The beauty of this tool is in its completeness and complexity. It is a route planner, a market database, a buy & sell location finder, and, most importantly, a complex trader calculating engine for multi-stop/multi-hop straight routes or loops. Until a few months ago, it was regularly updated by maddavo (http://www.davek.com.au/td/). Maddavo's market data (MMD) synced with EDDB daily, and was (semi-)manually updated by other tools (for example, EDMC can write station data in a format which can be updated to MMD). More importantly, MMD also had an EDDN listener. Therefore, MMD provided a full price list which was updated ebvery 24 hours, a rolline 48 hour last change list updated every 2 hours, and a rolling 3 hour new prices list updated regularly ("after a batch of data is processed"). There is a plugin for trade dangerous whoich can read these files, or they can be imported directly from the link on Maddavo's database as they are in the TD native prices" format. All was well in the world of the lite trader.

Unfortunately, over the past 4–6 months, maddavo's site has been almost completely static. Once in a whjile it updats for a day, and then remains quiet for weeks. My question to the coding experts and E:D lovers here is how difficult would it be to replicate/replace that functionality? Obviously this would have to be hosted somewhere too, but one step at a time.

Granted, that's probably asking a lot, just looking at the TD source code (including the plugins). As a hopefully more manageable question, how difficult would it be to be able to take a daily pricing download from EDDB or wherever, and munge it into a form which TD could import? I know that the listings.csv file on the API page of EDDB is not the native "prices" format; I've tried. Software that can be manually run to download data would be a big step! My minimal programing experience is in statistics (which is why I was able to upgrade the penalty function for distant stations); I know nothing about SQL or webscraping (and use R more than Python, although I do use it).

Any suggestions, or explanations as to why this is not a feasible task, would be greatly appreciated. Thank you.

CMDR Avi00013

Professionally speaking, why not ask Maddavo if you can take over his code? This would get you up to speed quickly, and the rest of the trade dangerous users, of course, and modifications could be made at a convenient time. the web site for maddavo's data is ow completely down (no DNS anymore) so I guess that Maddavo has abandoned the project.

Why reinvent the wheel?
 
Professionally speaking, why not ask Maddavo if you can take over his code? This would get you up to speed quickly, and the rest of the trade dangerous users, of course, and modifications could be made at a convenient time. the web site for maddavo's data is ow completely down (no DNS anymore) so I guess that Maddavo has abandoned the project.

Why reinvent the wheel?
I have reached out to him a number of times over the past few months; he has not responded :(
 
Replacing Maddavo's Database

I faced the same problem a couple of days ago, when I wanted to get back into trading, downloaded TD and realized that Maddavo's site was offline. So I started looking around for similar databases. I've found that EDDB makes all of its data available at https://eddb.io/api. Unfortunately it uses a different format than TD. I didn't have much time to play around with it, but I think it is certainly possible to write a script to parse the data from EDDB and convert it to the .prices format TD uses. The only problem I found was that the EDDB files use some ID numbers for systems and stations, while we need the actual names for the prices file. If this can be done, then it shouldn't be a problem to write. Of course this is only relevant if Maddavo is permanently down.
 
I've been thinking about generating my own DB for Trade Dangerous, originally so I could calculate intra-faction trading (i.e. just stations controlled by a player group). The best way to do this seemed to be a cut-down database with only the right stations in it :)

But the realities of that are that the station list changes over time, as the faction expands and retreats, wins and loses wars/elections. So I'd have to track more data in order to make sure I got the right set.

Taking a big data set and reducing it (e.g. the old maddavo data) took an unexpectedly long time, sqlite3 doesn't bulk delete that quickly! I guess similar things might happen with EDDB's dataset.

Listening to EDDN isn't very difficult, however, so I've been considering doing that - I've had other tools in the past that tracked influence changes, for example.

I've got server space, so it's do-able. How many of you are wanting this? Or has TD done its day? I'm not really a trader, so it isn't essential for my play ...
 
Top Bottom