In-Development TradeDangerous: power-user trade optimizer

@maddavo (and anyone interrested):

I've created a clone of tradedangerous on bitbucket and commited my changes for the "update" command.

  • changes to the "update" command:
    • new [--all] switch - Generates the temporary file with all columns and new timestamp.
    • if the station is unknown it will generate a list of all items with "0" prices/demand/stock

Q: How do I get it?
A: Use git (add my repository and pull from it)

Code:
$ git remote add bgol https://bitbucket.org/bgol/tradedangerous.git
$ git pull bgol update_all_switch

I've tagged the changes, so only this will get merged.

This has been merged into 4.1, thank you!
 
Hello guys.

Code:
C:\td>trade.py run --sh hauler --fr chan --cr 20000
Traceback (most recent call last):
  File "C:\td\trade.py", line 936, in <module>
    main()
  File "C:\td\trade.py", line 925, in main
    tdb = TradeDB(debug=args.debug, dbFilename=args.db)
  File "C:\td\tradedb.py", line 344, in __init__
    self.reloadCache()
  File "C:\td\tradedb.py", line 411, in reloadCache
    data.buildcache.buildCache(dbPath=self.dbPath, sqlPath=self.sqlPath, pricesPath=self.pricesPath)
  File "C:\td\data\buildcache.py", line 156, in buildCache
    processPricesFile(tempDB, pricesPath)
  File "C:\td\data\buildcache.py", line 118, in processPricesFile
    for price in priceLineNegotiator(pricesFile, db, debug):
  File "C:\td\data\buildcache.py", line 99, in priceLineNegotiator
    itemID = itemsByName["{}:{}".format(categoryID, itemName)] if qualityItemWithCategory else itemsByName[itemName]
KeyError: 'Battle Weapons'

Anyone able to give me a hand?
 

Harbinger

Volunteer Moderator
Slight fix to my previous coordinate submission to make them 1/32 compliant:
Code:
INSERT INTO "System" VALUES(327,'Perkwunos',-40.5,46.75,6.75,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(328,'LP 274-8',-29.75,44.9375,20.3125,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(329,'Rho Coronae Borealis',-29.875,42.09375,22.21875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(330,'LP 386-49',-25.53125,33.375,26,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(331,'LP 329-18',-26.625,39.875,23.28125,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(332,'LHS 3124',-26.28125,43.4375,24.1875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(333,'LHS 399',-13.5625,36.15625,25.78125,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(334,'LHS 396',-9.875,30.84375,20.46875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(335,'OT Serpentis',-11.125,30.34375,18.40625,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(336,'DE Bootis',-7.4375,32.625,17,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(337,'Veren''s Stop',-12.96875,21.96875,20.28125,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(338,'G 181-6',-20.3125,20.3125,14.0625,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(339,'Marcov''s Point',-22.34375,14.625,17.65625,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(340,'LP 229-17',-23.03125,9.03125,8.96875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(341,'G 203-51',-15.875,12.03125,5.34375,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(342,'Bidmere',-7.4375,13.21875,1.90625,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(343,'Wise 1405+5534',-8.03125,13.4375,-1.84375,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(344,'LHS 455',-16.90625,10.21875,-3.4375,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(345,'LP 71-165',-20.90625,11.09375,-2.21875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(346,'LHS 465',-23.9375,12.21875,-0.625,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(347,'Austern',-25.15625,15.34375,9.375,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(348,'G 202-48',-15.53125,14.375,1.875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(349,'2MASS 1503+2525',-6.1875,18,8.125,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(350,'DG Canum Venaticorum',-3.125,25.53125,2.6875,102,'2014-10-15 00:00:00');
INSERT INTO "System" VALUES(351,'LP 378-541',1.1875,20.71875,2.34375,102,'2014-10-15 00:00:00');
 

ShadowGar

Banned
Are all these forks and branches reintegrated into the master at some point?
I fail go get an impression what is the most feature- and data-complete version.

Yes. Just download the main one. Everyone is writing their code in individual branches and main is merging them.
 
Smacker's "Local" command has been merged into 4.2.0: It allows you to query the systems (and/or stations) around you.

Code:
osmith@WOTSIT /c/dev/trade (master)
$ trade.py local --ship=side aulin
Local systems to AULIN within 8.13 ly.
--------------------------------------
 3.53 i Bootis
 6.55 Rakapila
 6.98 Styx
 7.16 BD+47 2112
 7.74 LHS 3006
 7.75 Eranin
 7.80 Opala

osmith@WOTSIT /c/dev/trade (master)
$ trade.py -v local --ship=side aulin
Local systems to AULIN within 8.13 ly.
--------------------------------------
 3.53 i Bootis
        <Chango Dock>
        <Maher Stellar Research>
 6.55 Rakapila
        <Stone Enterprise>
 6.98 Styx
        <Bixby Station>
        <Wingrove Platform>
 7.16 BD+47 2112
        <Olivas Settlement>
 7.74 LHS 3006
        <WCM Transfer Orbital>
 7.75 Eranin
        <Azeban City>
        <Azeban Orbital>
        <Eranin 4 Survey>
 7.80 Opala
        <Romanenko Estate>
 
trade.py web interface

I'm tinkering with a web interface passing args to trade.py in the background. I have managed to get trade.py to output the -h help and the run -h help on a web page so I know that python runs it OK.

But when I try to get it to execute the run command with actual args then I get the following output:
Code:
Traceback (most recent call last):
File "C:\[REDACTED]\TradeDangerous\trade.py", line 1008, in 
main()
File "C:\[REDACTED]\TradeDangerous\trade.py", line 997, in main
tdb = TradeDB(debug=args.debug, dbFilename=args.db)
File "C:\[REDACTED]\TradeDangerous\tradedb.py", line 344, in __init__
self.reloadCache()
File "C:\[REDACTED]\TradeDangerous\tradedb.py", line 411, in reloadCache
data.buildcache.buildCache(dbPath=self.dbPath, sqlPath=self.sqlPath, pricesPath=self.pricesPath, debug=self.debug)
File "C:\[REDACTED]\TradeDangerous\data\buildcache.py", line 152, in buildCache
with sqlPath.open() as sqlFile:
File "C:\python34\lib\pathlib.py", line 1070, in open
opener=self._opener)
File "C:\python34\lib\pathlib.py", line 944, in _opener
return self._accessor.open(self, flags, mode)
File "C:\python34\lib\pathlib.py", line 323, in wrapped
return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: 'data\\TradeDangerous.sql'

I can run the command on the web server console OK so I think it must be some kind of permissions thing. But I have set permissions for the TradeDangerous directory to "Full Control" to the Everyone user in IIS. Anyone know why this would be?
 

wolverine2710

Tutorial & Guide Writer
Nice, I added them to my fork. I have made a plot here

Got some other coords from JesusFreke and Wolverine2710.

Got up to 386 systems now!

Great. Not a gitter (cvs, svn previously) but have checked your fork. Currently tradedangerous.sql contains 366 systems. Two are commented out so 364 active sytems. I assume your busy processing the rest?

In the issues list of kfsone's master you mentioned that you have added my crowdsourced systems. I can't find them in the .sql file. I know the ''What is the most efficient way to crowdsource the 3D sytem coordinates for Beta 2 ? " thread is getting lots of posts but you seem to have missed the data I collected on saturday. They are in post #262.

I hope the dust settles soon in my thread and we can start with an automatized 'crowd sourcing effort' in such a way that it will be easier to sync the calculated data with TD. Thanks for all your effort so far. Very much appreciated.

Edit: Tried to add it to the issues comment in the master kfsone repository also but when I previewed my message something did go wrong and it was lost. Hence only here, my bad.
 
Last edited:
Can someone catch me up on the progress of Trade Dangerous?

Have been busy recently..

If I understand correctly... the program still works.. except for the auto updates of the actual commodity prices.
Which has to be completed manually??

Also.. work is still in progress of adding new systems and their commodities?

Is there any hope in a screen cap of the commodity then transferred to a file to be shared?


oh...
 
Looks a fantastic tool. Is there any chance of embedding it into a GUI for ease of use?

Edit. Btw, love the use of the X52 MFD screen.
 
Last edited:
As a player with very little programming experience, am finding it difficult to work out how to update prices.

I tried running something like

trade.py update "chango" --editor "C:\Program Files (x86)\Notepad++\notepad++.exe" --all

from the cmd prompt and get a prices.tmp file in notepad++.

The problem is what to do next. Not sure how to edit the prices and have it update the main price list. Any help would be much appreciated.
 
Great. Not a gitter (cvs, svn previously) but have checked your fork. Currently tradedangerous.sql contains 366 systems. Two are commented out so 364 active sytems. I assume your busy processing the rest?

In the issues list of kfsone's master you mentioned that you have added my crowdsourced systems. I can't find them in the .sql file. I know the ''What is the most efficient way to crowdsource the 3D sytem coordinates for Beta 2 ? " thread is getting lots of posts but you seem to have missed the data I collected on saturday. They are in post #262.

I hope the dust settles soon in my thread and we can start with an automatized 'crowd sourcing effort' in such a way that it will be easier to sync the calculated data with TD. Thanks for all your effort so far. Very much appreciated.

Edit: Tried to add it to the issues comment in the master kfsone repository also but when I previewed my message something did go wrong and it was lost. Hence only here, my bad.
No, I missed those 3, will add them now..... Then I'll put a merge request back to The One True Repository
 

wolverine2710

Tutorial & Guide Writer
No, I missed those 3, will add them now..... Then I'll put a merge request back to The One True Repository

That would be great. Hopefully "the One" is able to add it to TOTR when he has time. That was tongue in cheek ;-) In the crowd source effort we currently don't have a masterlist of system-coords - which is a headache. I'm hoping that tradedangerous.sql can be our reference till that happens. Also haven't checked but Harbinger yesterday updated his coords so that they are in 1/32LY grid. Have you taken that update into account?
 
Last edited:

wolverine2710

Tutorial & Guide Writer
As a player with very little programming experience, am finding it difficult to work out how to update prices.

I tried running something like

trade.py update "chango" --editor "C:\Program Files (x86)\Notepad++\notepad++.exe" --all

from the cmd prompt and get a prices.tmp file in notepad++.

The problem is what to do next. Not sure how to edit the prices and have it update the main price list. Any help would be much appreciated.

Notepad++ might not be the right choice - by default. It had the same problem as you. See post #150 and the responses after that. It will tell you everything you need to know. Also check out the new --all switch for the update command wich got merged recently. Makes life easier, see post #222.
 
Last edited:

wolverine2710

Tutorial & Guide Writer
Can someone catch me up on the progress of Trade Dangerous?

Have been busy recently..

If I understand correctly... the program still works.. except for the auto updates of the actual commodity prices.
Which has to be completed manually??

Also.. work is still in progress of adding new systems and their commodities?

Is there any hope in a screen cap of the commodity then transferred to a file to be shared?


Your own "oh...." reply won't give you your answesr. The answers you are looking for can be found here and in the forums but I admit it can be a bit daunting to find it. To answer your questions:

Q: If I understand correctly... the program still works.. except for the auto updates of the actual commodity prices. Which has to be completed manually??

A: Correct. TD is being actively updated as well.

Q: Also.. work is still in progress of adding new systems and their commodities?
A: The thread "What is the most efficient way to crowdsource the 3D sytem coordinates for Beta 2?". The coords found are fed into TD.

Q: Is there any hope in a screen cap of the commodity then transferred to a file to be shared?

A: The thread: "Has any one experimented with OCR tools for scraping the commodities market prices? ". Aside from some proof of concepts nothing solid exists atm. If/when OCR is viable its a matter of setting up some sort of phoenix EMDN and emdn-tap.py can read its data. Don't expect anything soon though.
 
Last edited:

wolverine2710

Tutorial & Guide Writer
Yes, his updates are in.

Perfecto.

Not quite sure about how TD works. Would it be possible just to fetch your tradedangerous.sql file and run sqllite3 on it or would that destroy data in the database - like prices info?

I know I can just remove the .db file and when running TD it will recreate it. Does is then also transfer the data in the .prices file to the database. As in effectively I've a new database with your coords and my prices?
 

Harbinger

Volunteer Moderator
The actual prices are stored in prices.py I believe.

If you update tradedangerous.sql on the fly then it updates the systems you know about from the very next time you run a command via trade.py so replacing it with a new version means you know about new systems instantly.

As for the crowd sourcing, what if I set up my own fork of trade dangerous for my web form and any time a new system is mapped I get it to automatically commit changes to tradedangerous.sql on my fork along with a comment displaying a link to the data so it can be independently verified.

It would then be very simple for the new systems to be merged into the one true fork or anyone else to monitor my fork for updates.

I don't know if I'm just being a bit ambitious but I think it should be within the realms of possibility to do something like this.
 
Last edited:
Yes. Just download the main one. Everyone is writing their code in individual branches and main is merging them.

Thank you guys for the feedback.
Git intimidates me a little bit, I'm just used to centralized repos like CVS/SVN ^^

Thumbs up for being such an active bunch :)
 

wolverine2710

Tutorial & Guide Writer
The actual prices are stored in prices.py I believe.

If you update tradedangerous.sql on the fly then it updates the systems you know about from the very next time you run a command via trade.py so replacing it with a new version means you know about new systems instantly.

As for the crowd sourcing, what if I set up my own fork of trade dangerous for my web form and any time a new system is mapped I get it to automatically commit changes to tradedangerous.sql on my fork along with a comment displaying a link to the data so it can be independently verified.

It would then be very simple for the new systems to be merged into the one true fork or anyone else to monitor my fork for updates.

I don't know if I'm just being a bit ambitious but I think it should be within the realms of possibility to do something like this.

Thanks for the answers. Gonna try it.

Concerning crowd sourcing and git. Don't know your git and programming skills - as in how long it would take you. The last thing I want to do is discourage someone. Atm I'm interested in speed, which often means simplicity. We NEED to get our data input volunteers and make live easy for them.

Have you been able to talk to Red Wizard about automating things. See post #292 and post #314 of the "coords crowd sourcing" thread?

Lets assume that is implemented. Perhaps the following would suffice for the moment - speed. When someone enters data at your webpage or a json structure is uploaded by RW's tool. Update an insert.txt file. In it a timestamp, followed by an insert statement. No need to have the system_id incremented. Make it so that the file can be downloaded. The insert statement can be put into a tradedangerous.sql file of some fork - by copy/paste.

Concerning verifying distances entered on your site. Create a new .json file with name of entered station containing all entered data. Atm you and RW are using the same reference points. He could make a verify button on his entry.html page which reads in your data and verifies it. As his tool shows what distances are good or not.

Just my 2 eurocent. Again I LOVE the work you are willing to do for us all and don't want to discourage you and if you want to do it the way you suggested please DO go for it. I'm just asking to consider what I wrote. Mainly the speed thing.
 
Last edited:
Back
Top Bottom