In-Development TradeDangerous: power-user trade optimizer

[wolverine2710]

With regard to pull requests. It seems they both contain more then just the systems.csv file. Smackers systems.csv file seem to be up to date. I LIKE Smackers unit tests, smoketests very very much btw as it can make detecting regressions much easier. The multiple files make merging less easy for kfsone.

Smacker. As we probably all want the systems.csv to be up to date in TD asap would it be possible for you to create a pull request which ONLY has the systems.csv file. Under the assumption that the current 4.3.0 release already is using the systems.csv file it would allow kfsone to merge that one in TD quite easily. That said, not a gitter so don't know how feasible it is - as the systems.csv file already is in both pull requests.

Just my 2 euro cent - fingers crossed while writing this.

 

[kfsone]

Yes, nice work indeed! Using a format directly usable by outsiders certainly makes sense. Even an external utility like mine could have been done using the csv idea, perhaps with metadata (table file names and sql) in additional file, and optional zip/unzip support which is easy with python.

My concern was being able to make updates easier, and I thought it is easier to share updates in a single file. What I haven't realized is that the updates (and conflicts) happens mostly in the System table, so there is no real need to have a single import/export file for everything.

It will vary, game patches will generate flurries of updates in one file or another, splitting them up is better overall.

But more important is flexibility: I want to keep TD as an engine with core modules so that people can come along and develop nice Trade Engines using TD to power them.

So there's no reason not to incorporate a json reader: My intent is to add an "import" command which will fetch a bunch of new data into your current system without killing your current system if the data is bad.

Some of the thinking off the top of my stack:

  trade.py import incoming/Systems.json
  trade.py import https://foo.bar.com/Systems.Beta3.json
  trade.py import TradeDangerous.pkg

(The ".pkg" will be a file that uses a mime-like mechanism to do inline multipart files so you can ship several sub-files at once)

What's important about this compared to just dumping the files in your data directory?

Imagine you download "TradeDangerous.prices" from "Jhacker1"'s website. You drop it into your data directory.

  trade.py run --ship side --cr 1000 --from chango
  * Rebuilding DB Cache.
  * Error: OMG it's full of crap: data/TradeDangerous.prices
  * Error: All your data is gone.

In a nutshell, "import" will copy your existing DB to a new file, read the new values into that, and only when everything looks good will it import the new data back into your .db

 

[kfsone]

Is the knapsack-part optional? I remember that confused me a lot...

While that might be useful while funds are limited (and fun to implement ), once enough funds are available (and one does not spend his money on a new shiny ship too early) it just makes the output more complex. This is very subjective, but I preferred to see a list of possible trade routes nearby (not necessarily including the one I'm currently in) to choose from, and make an informed decision based on this list, depending on factors:

• route length
• visited stations
• age of price data
• availability of alternative commodities (should the one planned run out)
• current lunar phase or the mood I'm in

For best-trade calculation? No, the knapsack isn't optional. It's kind of the point

There is a "local" sub command that will show you nearby stations.

I'm sure we could add a "--sales" command to the "local" subcommand which tells you the #1 sale on that route.

 

[kfsone]

I've imported RedWizzard's additions/fixes into my own data/Systems.csv.

There are 62 additional stars plus several name fixes (zeros used instead of the letter "O" in some of the Wredguia systems etc.) I also corrected the position of SOL in the file as it should be at exactly 0,0,0 and not a calculated position.

As the changes are not just a simple addition I'm instead including a link to the modified csv file.

Much appreciated, Harbinger

 

[kfsone]

This is a nifty idea, but the interface is as clear as mud to me, I'd prefer a WIMP interface.

That will probably happen once we've achieved some stability with the underlying engine and data.

If you have questions, feel free to post.

 

[kfsone]

Wife agro limit received

Just merged Gazelle's big glut of changes.

Harbinger: If you're actively modifying the .csv files to sync us up, I'd be willing to explain how you can use a "fork" or branch of our repository to make your life easier If you're interested, email me: oliver@kfs.org

-Oliver

 

[fawick]

Is there a shared .prices available? And have you considered fetching the prices from elitetradingtool.co.uk?

 

[Harbinger]

Just merged Gazelle's big glut of changes.

Harbinger: If you're actively modifying the .csv files to sync us up, I'd be willing to explain how you can use a "fork" or branch of our repository to make your life easier If you're interested, email me: oliver@kfs.org

-Oliver

Yeah, it may be advisable for me to set up my own fork in future but we're pretty close to having the entire pill mapped now.

The System.csv in Smacker's development branch of TradeDangerous currently has the most complete and up to date collection of star systems and that is what I'm using as my point of reference:

https://bitbucket.org/Smacker65/tra...53538120e3259f/data/System.csv?at=development

In addition to that I've recently mapped a further 4 stars:

As we're nearing 570 stars in the pill it's getting very hard to locate the missing ones now but:

• Upsilon-1 Cephei (-87.09375, 13.875, -7.15625).
• Wolf 1080 (-77.8125, 12.84375, -5.78125).
• LTT 16470 (-64.375, 9.5, -19.625).
• Liabefa (-77, 39, -56).

'Upsilon-1 Cephei',-87.09375,13.875,-7.15625,'Harbinger','2014-10-19 19:06:35'
'Wolf 1080',-77.8125,12.84375,-5.78125,'Harbinger','2014-10-19 19:16:07'
'LTT 16470',-64.375,9.5,-19.625,'Harbinger','2014-10-19 19:26:28'
'Liabefa',-77,39,-56,'Harbinger','2014-10-19 20:12:43'

Verification

 

[Smacker]

With regard to pull requests. It seems they both contain more then just the systems.csv file. Smackers systems.csv file seem to be up to date. I LIKE Smackers unit tests, smoketests very very much btw as it can make detecting regressions much easier. The multiple files make merging less easy for kfsone.

Smacker. As we probably all want the systems.csv to be up to date in TD asap would it be possible for you to create a pull request which ONLY has the systems.csv file. Under the assumption that the current 4.3.0 release already is using the systems.csv file it would allow kfsone to merge that one in TD quite easily. That said, not a gitter so don't know how feasible it is - as the systems.csv file already is in both pull requests.

Just my 2 euro cent - fingers crossed while writing this.

OK working on it.
EDIT: Push sent, 567 Systems in the Pill

 

[Biteketkergetek]

I've run into a problem running TD (or really just the SQL of it) on my node running Ubuntu 12.04 LTS.

The python3 installation there has an sqlite version 3.7.9 that does not yet support the WITHOUT ROWID syntax.

This could be fixed with the following filter (I use this for my json import/export):

if sqlite3.sqlite_version < "3.8.2":
        sql = sql.replace("WITHOUT ROWID", "")

I wonder if there is a better solution for such older systems.

Additionally, AUTOINCREMENT could be removed as an optimization, too (its existence should cause no problem, though - apart from the slight performance decrease outlined in the link).

 

[Smacker]

I've run into a problem running TD (or really just the SQL of it) on my node running Ubuntu 12.04 LTS.

The python3 installation there has an sqlite version 3.7.9 that does not yet support the WITHOUT ROWID syntax.

This could be fixed with the following filter (I use this for my json import/export):

if sqlite3.sqlite_version < "3.8.2":
        sql = sql.replace("WITHOUT ROWID", "")

I wonder if there is a better solution for such older systems.

Additionally, AUTOINCREMENT could be removed as an optimization, too (its existence should cause no problem, though - apart from the slight performance decrease outlined in the link).

Best thing is to submit a ticket HERE

 

[Crazy_HK_Monkey]

Is anyone writing a front end for this excellent product?

 

[kfsone]

Is anyone writing a front end for this excellent product?

Actually, one of the forks is doing some amazing goodness with a thing called "Flask" to build a web based interface that runs on your own machine:

https://bitbucket.org/kfsone/tradedangerous/pull-request/19/flask-webapp-v011/diff

 

[kfsone]

Just merged Smacker65's data.

 

[wolverine2710]

Just merged Smacker65's data.

Perfect. It has all the latest changed discussed in the crowd source thread from today. Its in sync with THE reference sytems list maintained by RedWizzard.

The OP mentions as latest version 4.2.3. As we are now 4 version along (4.5.1) would it be possible for you to update the OP so it reflects the current state of TD?

 

[wolverine2710]

Actually, one of the forks is doing some amazing goodness with a thing called "Flask" to build a web based interface that runs on your own machine:

https://bitbucket.org/kfsone/tradedangerous/pull-request/19/flask-webapp-v011/diff

Noticed the pull request for it. Looks very VERY promising. Lately a LOT of new code is being created for TD. Stuff like unit and smoketests is awesome. Must be a nightmare to merge everything back into the master!!

 

[kfsone]

Noticed the pull request for it. Looks very VERY promising. Lately a LOT of new code is being created for TD. Stuff like unit and smoketests is awesome. Must be a nightmare to merge everything back into the master!!

It's all been pretty well done by these guys, the bottle neck is my available time.

 

[ShadowGar]

I've been working on a side project for TD. It is not as good as flask but for the simplier approach, I've set up a tester site for TD Web.

The idea behind this was with Overwolf now able to support overlays in ED. I have been writing a module for overwolf that will allow TD to run on an overlay. It basically will run a website locally and wrap it with overwolf.

This wasn't meant for full fledged interaction, like adding prices without tabbing. But you can still run commands from the site and get output. The backend is still the same program, python. This is just a way to talk to it locally on a pc.

I think that if we have different flavors of front ends, it will bring more in to appreciate the software. You can have the high end flask, the low end standard command line web, or the cmd window on desktop.


[Redacted site address - Program is Discontinued]

This is only a concept design at this point.

 

[Smacker]

I've been working on a side project for TD. It is not as good as flask but for the simplier approach, I've set up a tester site for TD Web.

The idea behind this was with Overwolf now able to support overlays in ED. I have been writing a module for overwolf that will allow TD to run on an overlay. It basically will run a website locally and wrap it with overwolf.

This wasn't meant for full fledged interaction, like adding prices without tabbing. But you can still run commands from the site and get output. The backend is still the same program, python. This is just a way to talk to it locally on a pc.

I think that if we have different flavors of front ends, it will bring more in to appreciate the software. You can have the high end flask, the low end standard command line web, or the cmd window on desktop.


Here is the site http://76.189.82.1:8080/form.php

This is only a concept design at this point.

Cool, nice logo!

 

[Harbinger]

2 New systems found by RedWizzard:

'Matask',-70.25,51.03125,-47.46875,'RedWizzard','2014-10-21 05:24:29'
'WREDGUIA IF-L b49-3',-88.375,35.40625,-4,'RedWizzard','2014-10-21 12:43:18'