Page 4 of 156 FirstFirst 1234567814 ... LastLast
Results 46 to 60 of 2330

Thread: TradeDangerous: power-user trade optimizer

  1. #46
    I've just published the first alpha of version 3.0:

    - Switched from the Microsoft Access DB to using text files as source and an SQLite database backend as a cache
    -- Laymans terms:
    Faster, easier to use, simpler to install,
    Edit "data/TradeDangerous.prices" with pricing updates,
    -- Dev notes:
    The .db file gets rebuilt if it's missing or if the .sql or .prices file is updated.

    - Should be possible to run on a Mac or a Linux box now (I haven't tested yet, will do that when my wife isn't wondering why I'm spending our first anniversary not even playing ED

    - Lots of cleanup,

    - Lots of speed improvements,


    If you're feeling frisky and want to give it a try, it's still in its own branch:

    https://bitbucket.org/kfsone/tradeda.../branch/sqlite


    Let me know if you run into any problems.

    Still haven't heard back from Saitek about driving the soft-buttons on the X52 through Python.

    -Oliver

  2. #47
    Ok, version 3.0 is live in the main branch now.

    It no-longer uses MS Access, it uses the batteries-included Python sqlite support instead. It builds the database from files in the "data" directory. If you want to update your local prices, just edit "TradeDangerous.prices" (should be self-explanatory) and next time you run the tool it will incorporate the data.

    A few of the command line options have been shuffled around, mostly to make it so that you can use abbreviations ("--cr" instead of "--credits", etc) and it has slightly more ship information.

    In the next couple of versions I'll be improving the checklist system to be more like a console that tracks where you're at, how much money you have, etc.

    I'm also planning a "--goal" option to let you say 'tell me the runs I need to make to get this much money', to make the MFD/Checklist more interactive (so if you have to correct prices after 2 hops, you can ask it to recalculate your route for you).

    Let me know if you have any problems.

  3. #48
    Awesome work! To be honest I don't understand how this algorithm is so fast compared to some other trade route finders :-)

    I'm in the process of writing a tool to pull data down from EMDN and loading it into tradedangerous.

    I started to pipe fh.exe's output (from Andreas' marketdump package) to my tool, parsing the json and producing your import format.

    As you switched to sqlite now, I think it would be more efficient to write to the db directly, so I would start again (and directly connecting to 0mq instead of piping from fh.exe). It is written in go, as I don't do python (I'm sorry...)

    I have the problem that Station and Item ID's do not match your schema because they are getting it directly from the engine, and I reckon the display names in the market screen differ sometimes. For example, Hel-Static Furnaces => heliostaticfurnaces.

    I think the best way to tackle this would be an extra column in the schema (i.e. emdn_name) for Items and Station/System Names. Would you take a patch for this? It would spare me from mainaining a fork :-)

    Thanks again,

    CMDR Lazerguns

  4. #49
    Originally Posted by Lazerguns View Post (Source)
    Awesome work! To be honest I don't understand how this algorithm is so fast compared to some other trade route finders :-)

    I'm in the process of writing a tool to pull data down from EMDN and loading it into tradedangerous.

    I started to pipe fh.exe's output (from Andreas' marketdump package) to my tool, parsing the json and producing your import format.

    As you switched to sqlite now, I think it would be more efficient to write to the db directly, so I would start again (and directly connecting to 0mq instead of piping from fh.exe). It is written in go, as I don't do python (I'm sorry...)

    I have the problem that Station and Item ID's do not match your schema because they are getting it directly from the engine, and I reckon the display names in the market screen differ sometimes. For example, Hel-Static Furnaces => heliostaticfurnaces.

    I think the best way to tackle this would be an extra column in the schema (i.e. emdn_name) for Items and Station/System Names. Would you take a patch for this? It would spare me from mainaining a fork :-)

    Thanks again,

    CMDR Lazerguns
    Oh, absolutely; grab the latest 3.1, and basically edit the data/tradedangerous.sql file and mail me a diff (oliver@kfs.org).

    3.x has some significant changes, and 3.1 more so. Really all your version needs is a variant of the cache populate function.

    And with regards to speed, it's lacking there, I need to do another optimize pass. I'm still learning python, and while I was able to use some nice tricks to stop python making it show, this is a pretty simple operation and very well suited to modern hardware. It really ought to be able to do several hundred hops per second rather than 10-20. But I'm still w writing my python like a c programmer

  5. #50
    Hey Mate,

    I thought I try out the version 3.0 but having some issues. I tried running a simple command.

    trade.py --from Chango --credits 20000 --hops 2

    And i get the following.

    usage: trade.py [-h] [--debug] [--db DB] {run,update} ...
    trade.py: error: argument subparser: invalid choice: 'Chango' (choose from 'run', 'update')

    if I just run trade.py, i get the following:

    ERROR: Specified file, "data\TradeDangerous.db", does not exist.

    Help?

    Thanks,
    Hondo

  6. #51
    Hey, Hondo,

    The command line has been expanded, as a result you have to tell it you want to do a trade run -- you'll see why this inconvenience in a version or two

    So instead of just "./trade.py ..." you have to say "./trade.py run".

    Assuming you have a Lakon Type 6, you can do:

    ./trade.py run/ --fr chango --cr 20000 --ship type6

    Things to note:

    . If in doubt, add "-h" to the line, e.g. "./trade.py -h" or "./trade.py run -h".

    . You don't *have* to type out the full form of each option - e.g. --fr instead of --from, --cr instead of --credits.

    . 2 is the default number of hops,

    . You need to specify your ship so it can approximate things like how far you can jump, ship names use the same matching code so type6 or "lakon type 6" or "pe 6" will match.

    . The only other command at the moment is "update", although I've only half implemented it.

    . The "db error" is my bad, it's supposed to tell you that you didn't select a command. I've checked in a fix for it.

  7. #52

  8. #53
    Other sub-commands I have in-mind:

    nav[igate] stationA stationB [--via ...] [--avoid ...]
    Plan jumps for me to get from A to B

    find ...
    Tell me the best closest or best place to buy items,
    ships or upgrades from where I am; or find the best
    place to sell a cargo I have.

  9. #54
    Thanks Mate,

    It works like a charm.

    I got around the Saitek MFD using autohotkey (http://www.autohotkey.com/)

    With that app and help form their members on the forums I now use a script that when I press enter it send the enter command to the cmd and I can step through the every step in the MFD without having to all tab out. Its only Temp fix but it works.... hopefully you will hear back from saitek so there can be a permanent fix.

  10. #55
    Originally Posted by Hondo View Post (Source)
    Thanks Mate,

    It works like a charm.

    I got around the Saitek MFD using autohotkey (http://www.autohotkey.com/)

    With that app and help form their members on the forums I now use a script that when I press enter it send the enter command to the cmd and I can step through the every step in the MFD without having to all tab out. Its only Temp fix but it works.... hopefully you will hear back from saitek so there can be a permanent fix.
    Apparently it's "complex", and it may take them some time to get an answer (their words).

  11. #56
    Originally Posted by Lazerguns View Post (Source)
    I'm in the process of writing a tool to pull data down from EMDN and loading it into tradedangerous.
    I knocked up a small EMDN module in python to do this, grab version 3.3. I've also added support for the internal game names (table is AltItemName in the DB).

    -Oliver

  12. #57
    I haven't used your tool, but after a 2 week vacation going to use/test it. Very curious how it compares to BPC wrt speed. If I've interpreted it correctly it now uses sqllite to read its data instead of Access. Cool. I plan to install it on my 2nd PC which is on 24/7. Is it possible to run the tool on my ED PC and get the data from my 2nd PC?

    Not tested it, but is TD multi-threaded?
    If not, is that something your are planning to add?
    You have a navigation system in place (possible using Dykstra or an A* search. Atm there are two navigation systems out there which also take fuel consumption into account. Is that something you are planning to integrate into TD?

    Very curious what your tool will contain when I'm back in two weeks ;-)
    I'm a CLI person, so I very much like this tool. Also I happen to have a X52!!!

    Edit: Out of your hands but something for Andeas. Because of missing securirty setups in the ElasticSearch Andreas can't have users accessing it directly. The firehose works fine when you contact to a it for a longer period. Would be nice if Anderas would have a web api which gives back the latest prices for a station, multiple stations, or even all stations. That preserves band with on his side I think and it would be possible to directly have the latest prices, for for example TD. That EMDN of his works great and enables authors to have data without setting up hosted databases themselves. Tip of the iceberg I think of what would be possible!!
    Even better if Andreas somehow finds a way to protect ES and have it read-only open to the world so that ES-queries can be made directly. There are tools for this I believe....

    Edit: Good luck with your new home ;-)

  13. #58
    Abount EMDN. Kfsone, do you plan to integrate price update from EMDN into TD? I tried to somehow do it myself, but wasn't able to

  14. #59
    Originally Posted by Net-burst View Post (Source)
    Abount EMDN. Kfsone, do you plan to integrate price update from EMDN into TD? I tried to somehow do it myself, but wasn't able to
    From the OP: "v3.1 Sep 01/2014
    Introduced sub-commands:
    - "run" command provides the old default behavior of calculating a run,
    - "update" command provides ways to update price database easily,"

    Also from Andreas marketdump thread he post this. So it looks to be already in TD - to some extend. Though its not really documented. Tbh I've tried it but got an error about zeroMQ missing. Could be doing something wrong though.

    Perhaps in the future he's going to do it like Andreas, an exe with Python embedded into it. TD is blindingly fast. I can see this becoming an engine being used by others. As in others who create a GUI, call TD and then display the results in their GUI. Only thing missing for that is some optional JSON and/or XML output which can be easily parsed.

  15. #60
    Originally Posted by wolverine2710 View Post (Source)
    I haven't used your tool, but after a 2 week vacation going to use/test it....
    No, TD is not multi-threaded, there's really no reason for it to be, it's pretty fast... Here it's calculating a 30 hop run for a type 6 starting off with 600 credits all the way to 2.3 million:

    Code:
    $ time ./trade.py run --hops 30 --ship type6 --credits 600
    unspecified hops 31, numHops 30, viaStations 0
    LHS 417 Gernhardt Camp -> BOLG Moxon's Mojo:
     >-> At LHS 417/Gernhardt Camp, Buy: 54 x Biowaste,
      +  At WYRD/Vonarburg Co-operative, Buy: 96 x Leather,
      +  At TILIAN/Maunder's Hope, Buy: 100 x Basic Medicines,
      +  At BOLG/Moxon's Mojo, Buy: 100 x Mineral Extractors,
      +  At ITHACA/Hume Depot, Buy: 100 x Uraninite,
      +  At ACIHAUT/Cuffey Plant, Buy: 87 x Lithium, 13 x Hydrogen Fuels,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 99 x Reactive Armor, 1 x H.E. Suits,
      +  At LFT 880/Baker Platform, Buy: 100 x Uranium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Reactive Armor,
      +  At LFT 880/Baker Platform, Buy: 100 x Uranium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 73 x Consumer Tech, 27 x H.E. Suits,
      +  At LFT 880/Baker Platform, Buy: 100 x Uranium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Consumer Tech,
      +  At LFT 880/Baker Platform, Buy: 94 x Beryllium, 2 x Uranium,
      +  At ASELLUS PRIMUS/Beagle 2 Landing, Buy: 100 x Resonating Separators,
      +  At ACIHAUT/Cuffey Plant, Buy: 100 x Lithium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Consumer Tech,
      +  At LFT 880/Baker Platform, Buy: 100 x Beryllium,
      +  At ASELLUS PRIMUS/Beagle 2 Landing, Buy: 100 x Resonating Separators,
      +  At ACIHAUT/Cuffey Plant, Buy: 100 x Lithium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Consumer Tech,
      +  At LFT 880/Baker Platform, Buy: 100 x Beryllium,
      +  At ASELLUS PRIMUS/Beagle 2 Landing, Buy: 100 x Resonating Separators,
      +  At ACIHAUT/Cuffey Plant, Buy: 100 x Lithium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Consumer Tech,
      +  At LFT 880/Baker Platform, Buy: 100 x Beryllium,
      +  At ASELLUS PRIMUS/Beagle 2 Landing, Buy: 100 x Resonating Separators,
      +  At ACIHAUT/Cuffey Plant, Buy: 100 x Lithium,
      +  At LHS 3262/Louis De Lacaille Prospect, Buy: 100 x Consumer Tech,
      +  At LFT 880/Baker Platform, Buy: 100 x Beryllium,
     <-< Moxon's Mojo gaining 2,385,195cr => 2,385,795cr total
    drumroll:

    Code:
    real    0m2.800s
    user    0m0.000s
    sys     0m0.046s
    So there's really not a lot of reason for it to be multi-threaded.

    And it's been a long time since I did an optimization pass. I can probably drop 0.8-1.0s of that time if I give it an hour or so.

    Despite the speed, the algorithm is accurate, not just precise. If you look at the code in tradecalc.py you'll see there are two "fit" routines. One uses a sheer, bloody minded "try EVERY combination" approach and the other is the normal algorithm.

    In terms of jump distances and fuel costs, I'm already in the process of incorporating those in (check ships.py), as well as how far stations are from the landing point in a star system.

    And I just committed a version with the "emdn-tap.py" tool for fetching data from EMDN into the TD database.

Page 4 of 156 FirstFirst 1234567814 ... LastLast