You really didn't have to, I was just curious.
In-Development TradeDangerous: power-user trade optimizer
- Thread starter kfsone
- Start date
You really didn't have to, I was just curious.
I have no idea why Apache would intermittently send zero length in the headers and my google-fu is failing me.
If anyone has a suggestion, I would be happy to look at it.
EDIT: Looks like there might be a thing with mod_deflate where it can decide to NOT send a content-length header, on account of it's chunking up the data and doesn't know how long it will be.
EDIT 2: Correction. It's definitely a thing. If the server chunks the data, then we get a "Transfer-Encoding: chunked" and no "Content-Length". Why this only is an issue some of the time is a mystery.
FINAL EDIT: Discussed with Eyeonus - he will make a suitable change in transfers.py - Issue raised on github.
Last edited:
Eyeonus - Are you sure your default LSP took? And is it effective at that level?
Two eyewateringly long deliveries here for very moderate profit levels (hop 2 & hop 6):
Two eyewateringly long deliveries here for very moderate profit levels (hop 2 & hop 6):
Code:
D:\Games\Game Tools\Elite\Trade Dangerous>trade.py run --hops 7 --from hip15266/linnehan --to cubeo -vvv --pla=YN? --credits=95000000 --capacity=150 --ly-per=27.98 --empty-ly=32.03 --insurance=15500000 --unique --progress --pad-size=L --age=2
* Hop 1: .........1 origins
* Hop 2: ........87 origins .. 1,350-255,600cr gain, 9-1,704cr/ton
NOTE: Pruned 166 origins too far from any end stations
* Hop 3: .......376 origins .. 28,350-747,300cr gain, 94-2,491cr/ton
NOTE: Pruned 808 origins too far from any end stations
* Hop 4: .......575 origins .. 143,802-1,128,750cr gain, 323-2,508cr/ton
NOTE: Pruned 1845 origins too far from any end stations
* Hop 5: .......465 origins .. 368,550-1,276,200cr gain, 614-2,285cr/ton
NOTE: Pruned 1676 origins too far from any end stations
* Hop 6: .......350 origins .. 430,088-1,572,600cr gain, 573-2,096cr/ton
NOTE: Pruned 1066 origins too far from any end stations
* Hop 7: .......120 origins .. 983,100-1,821,201cr gain, 1,092-2,267cr/ton
HIP 15266/Linnehan Orbital -> Cubeo/Medupe City (score: 2072017.088780)
Start CR: 79,500,000
Hops : 7
Jumps : 14
Gain CR : 2,073,450
Gain/Hop: 296,207
Final CR: 81,573,450
Load from HIP 15266/Linnehan Orbital (2.00Kls, BMk:Y, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Medicines/Basic Medicines 322cr vs 1,786cr, <1 hr vs 8 hrs, total: 48,300cr
Unload at Kehperagwe/Carson Terminal (18ls, BMk:Y, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 219,600cr (1,464cr/ton) => 95,219,600cr
Load from Kehperagwe/Carson Terminal (18ls, BMk:Y, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Textiles/Military Grade Fabrics 1,031cr vs 3,414cr, 8 hrs vs 1 hr, total: 154,650cr
Unload at Witchhaul/Boodt Keep (169Kls, BMk:N, Pad:L, Plt:Y, Shp:Y, Out:Y, Ref:Y) => Gain 357,450cr (2,383cr/ton) => 95,577,050cr
Load from Witchhaul/Boodt Keep (169Kls, BMk:N, Pad:L, Plt:Y, Shp:Y, Out:Y, Ref:Y):
150 x Weapons/Non-lethal Weapons 1,438cr vs 1,766cr, 1 hr vs 12 hrs, total: 215,700cr
Unload at Kappa Fornacis/Harvestport (918ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 49,200cr (328cr/ton) => 95,626,250cr
Load from Kappa Fornacis/Harvestport (918ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Slavery/Imperial Slaves 14,231cr vs 17,324cr, 12 hrs vs 17 hrs, total: 2,134,650cr
Unload at Putamasin/Weil Orbital (1.65Kls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 463,950cr (3,093cr/ton) => 96,090,200cr
Load from Putamasin/Weil Orbital (1.65Kls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Textiles/Military Grade Fabrics 934cr vs 3,557cr, 17 hrs vs 20 hrs, total: 140,100cr
Unload at Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 393,450cr (2,623cr/ton) => 96,483,650cr
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Metals/Gold 9,304cr vs 10,820cr, 20 hrs vs 34 hrs, total: 1,395,600cr
Unload at LTT 537/Gurshtein Hub (395Kls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 227,400cr (1,516cr/ton) => 96,711,050cr
Load from LTT 537/Gurshtein Hub (395Kls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
150 x Consumer Items/Consumer Technology 5,862cr vs 8,278cr, 34 hrs vs <1 hr, total: 879,300cr
Unload at Cubeo/Medupe City (332ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 362,400cr (2,416cr/ton) => 97,073,450cr
----------------------------------------------------------------------------
Finish at Cubeo/Medupe City (332ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) gaining 2,073,450cr (1,974cr/ton) => est 97,073,450cr total
Last edited:
Yes, I am sure the default lsp is set. No, I am not sure it is effective at that level.
This is why I have said repeatedly that we need testing to determine the proper default, and the "recommended min and max" values you asked for to put in the documentation.
One or more people needs to do a bunch of different runs, at varying levels of lsp, and show the point(s) at which the resultant route changes, as I did with the route example that changes at lsp 3.76.
Once we have enough examples, we can figure out what a good default setting would be.
The easiest way to get the needed data is:
If you (as in anyone reading this post) encounter a run result with one or more long distance hops in it, re-run the route with different lsp values until you see a different route result.
My recommendation is to run lsp at 0, 25, 50, 75, 100, and then pinpoint the route change points from there.
In my case, there was only one route point change, but that's probably atypical. Their could be as many as a dozen change points, and I'm betting in most cases 4 or 5, although, admittedly, I'm guessing and could easily be wrong.
Last edited:
I will feel suitably chastised and do a bit of testing
So per my route above, the minimum to remove the 395k hop was LSP=12.
The 169k hop disappeared at LSP=16.
For the hop later in the list early destinations were unaffected. It seemed like other destinations only changed to accommodate the changes to the distant stations.
Last edited:
I will feel suitably chastised and do a bit of testing
So per my route above, the minimum to remove the 395k hop was LSP=12.
The 169k hop disappeared at LSP=16.
For the hop later in the list early destinations were unaffected. It seemed like other destinations only changed to accommodate the changes to the distant stations.
Okay, I've just realized that this is a bad place to be putting test runs. After I've had some sleep, I'll create an issue on eye-TD specifically for compiling the data, as well as a recommended format for submissions.
I don't know if you guys have experienced this - but the new simpleFit seems to be sending me to a greater variety of places. I generally use --unique anyway, but even so I am ending up at new places and especially more planetary landings than the old algorithm.
Which speaks nothing to the level of profit I might be making in comparison to the old method, but it is definitely more interesting - and that's worth a chunk of credits on its own.
Which speaks nothing to the level of profit I might be making in comparison to the old method, but it is definitely more interesting - and that's worth a chunk of credits on its own.
Probably because in order to run the old version and have it finish, we had to cut out so many options by passing max age or distance or supply etc. Now, the approximation is fast enough to explore a much larger universe of options, which is what you're seeing [cool].
It is finishing *MUCH* quicker, but I'm not really passing it any different parameters than before, if I'm honest.
Okay, everyone, I've got the data collection issue up for eye-TD.
Put your data here:
https://github.com/eyeonus/Trade-Dangerous/issues/11
Make sure to read the OP first.
Put your data here:
https://github.com/eyeonus/Trade-Dangerous/issues/11
Make sure to read the OP first.
I realise it's early days, but we basically have:
4, 8.5, 12 & 16
As relevant values. I hope the rest of the data starts to home in a bit better than that. We'll see I guess
I realise it's early days, but we basically have:
4, 8.5, 12 & 16
As relevant values. I hope the rest of the data starts to home in a bit better than that. We'll see I guess
The goal is to find a value that in the majority of cases does a reasonable job of not returning routes with horrendously long distance hops, like for instance the one you had with two.
I would say that a value that may occasionally have one ld-hop is okay, so in your case the lsp=12 would be reasonable. But it should at least 75% of the time not have any hops with >3500ls to travel.
I'm fairly certain just taking all the numbers we get from the submitted data and applying a weighted average will give us a decent default. And we can find effective min and max with the data as well, although, honestly, I'm kind of depending on our resident mathematician for that bit.
The beta testing version of TD Helper that works with EDDBlink is now available if anyone is interested in testing it. Setup instructions may be found at https://github.com/MarkAusten/TDHelper/wiki/Set-up-for-Beta-Testing.
I'm not going to start a separate thread for this until all of the component parts are production ready, which is going to be fairly soon judging by the recent posts.
Thanks to all, especially Eyeonus, Tromador and AVI0013 for writing, debugging and improving the TD/EDDBlink applications.
I'm not going to start a separate thread for this until all of the component parts are production ready, which is going to be fairly soon judging by the recent posts.
Thanks to all, especially Eyeonus, Tromador and AVI0013 for writing, debugging and improving the TD/EDDBlink applications.
Been playing with TD and noticed an oddity. Ran a route with 3 days as the max price age. Data as at 18:15 BST.
Results
So ran it with 7 days:
Which is odd since a limit of three days found nothing but the results show that the prices are only 2 days old. Ran a check with 4 days:
Got the same result as the 7 day limit.
Looks like the algorithm is not using the age of the price data correctly. Off by one error coupled with a < instead of <= somewhere?
Results
-u "trade.py" run --fr="TURIR/Brooks Terminal" --cap=720 --ins=3172226 --cr=430000000 --ly=24.73 --empty=37.61 --pad=L --margin=0.1 --hops=2 --jum=3 --age=3 --avoid="slaves,narcotics" --loop --summary --progress -vv
* Hop 1: .........1 origins
* Hop 2: .......490 origins .. 149,040-3,222,000cr gain, 207-4,475cr/ton
trade.py: Error: No profitable trades matched your critera, or price data along the route is missing.
* Hop 1: .........1 origins
* Hop 2: .......490 origins .. 149,040-3,222,000cr gain, 207-4,475cr/ton
trade.py: Error: No profitable trades matched your critera, or price data along the route is missing.
So ran it with 7 days:
-u "trade.py" run --fr="TURIR/Brooks Terminal" --cap=720 --ins=3172226 --cr=430000000 --ly=24.73 --empty=37.61 --pad=L --margin=0.1 --hops=2 --jum=3 --age=7 --avoid="slaves,narcotics" --loop --summary --progress -vv
* Hop 1: .........1 origins
* Hop 2: .......797 origins .. 68,400-3,222,000cr gain, 95-4,475cr/ton
Turir/Brooks Terminal -> Turir/Brooks Terminal (score: 1658189.998723)
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)
----------------------------------------------------------------------------
Finish at Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) gaining 3,309,120cr (2,298cr/ton) => est 433,309,120cr total
* Hop 1: .........1 origins
* Hop 2: .......797 origins .. 68,400-3,222,000cr gain, 95-4,475cr/ton
Turir/Brooks Terminal -> Turir/Brooks Terminal (score: 1658189.998723)
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)
----------------------------------------------------------------------------
Finish at Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) gaining 3,309,120cr (2,298cr/ton) => est 433,309,120cr total
Which is odd since a limit of three days found nothing but the results show that the prices are only 2 days old. Ran a check with 4 days:
-u "trade.py" run --fr="TURIR/Brooks Terminal" --cap=720 --ins=3172226 --cr=430000000 --ly=24.73 --empty=37.61 --pad=L --margin=0.1 --hops=2 --jum=3 --age=4 --avoid="slaves,narcotics" --loop --summary --progress -vv
* Hop 1: .........1 origins
* Hop 2: .......605 origins .. 68,400-3,222,000cr gain, 95-4,475cr/ton
Turir/Brooks Terminal -> Turir/Brooks Terminal (score: 1658189.998723)
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)
----------------------------------------------------------------------------
Finish at Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) gaining 3,309,120cr (2,298cr/ton) => est 433,309,120cr total
* Hop 1: .........1 origins
* Hop 2: .......605 origins .. 68,400-3,222,000cr gain, 95-4,475cr/ton
Turir/Brooks Terminal -> Turir/Brooks Terminal (score: 1658189.998723)
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)
----------------------------------------------------------------------------
Finish at Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) gaining 3,309,120cr (2,298cr/ton) => est 433,309,120cr total
Got the same result as the 7 day limit.
Looks like the algorithm is not using the age of the price data correctly. Off by one error coupled with a < instead of <= somewhere?
Last edited:
It shouldn't make any difference if the requested age is 3. The age of 3 finds nothing but 4 days find the 2 day old prices.
My first hypothesis are that since python is a zero index language, where the first item in an array or collection is the zeroth element, there might be what is known as an off-by-one error. The second hypothesis is that "less than" was used as an operator instead of "less than or equal to". If the price age was rounded to the next highest integer, then 2.1 days would round up to 3 and 3 < 3 is false whereas 3 <= 3 is true.
Only possibilities of course, there will be plenty of other reasons why this might happen. Formatting error on the output, for example where the calcs are correct but the calculation of the days for display is at fault.....
Last edited:
Server headers issue
Regarding the problem that Gurunot reported and which I am being told happens on occasion to everyone (apart from me, so far) where TD refuses to download files because of some kind of header problem.
"Exception: Remote server replied with invalid content-length."
Eyeonus has added some debug code, such that in that circumstance, TD will give a dump of the full headers it received from the server.
It would be helpful to me if you guys would pass that information on when it happens.
We have an issue open on github, which is probably the best place for any useful information - https://github.com/eyeonus/Trade-Dangerous/issues/10
If you don't want to use github, feel free to send me a pm.
Regarding the problem that Gurunot reported and which I am being told happens on occasion to everyone (apart from me, so far) where TD refuses to download files because of some kind of header problem.
"Exception: Remote server replied with invalid content-length."
Eyeonus has added some debug code, such that in that circumstance, TD will give a dump of the full headers it received from the server.
It would be helpful to me if you guys would pass that information on when it happens.
We have an issue open on github, which is probably the best place for any useful information - https://github.com/eyeonus/Trade-Dangerous/issues/10
If you don't want to use github, feel free to send me a pm.
Not too sure what is going on here, the result of the trade command
is the following:
But the market information for these stations for Gold and Military Grade Fabrics is this:
The route says to pick up Military Grade Fabrics from Brooks Terminal but the market output says that there are none. Likewise for God at Fortress York.
It looks to me as though the route information is the wrong way round and it should be:
Or is it just late on a Tuesday evening and the heat is affecting my brain cell?
Code:
-u "trade.py" run --fr="TURIR/Brooks Terminal" --cap=720 --ins=3172226 --cr=430000000 --ly=24.73 --empty=37.61 --pad=L --margin=0.1 --hops=2 --jum=3 --age=3 --avoid="slaves,narcotics" --loop --summary --progress -vvis the following:
Code:
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)But the market information for these stations for Gold and Military Grade Fabrics is this:
Code:
trade.py market "TURIR/Brooks Terminal" -vv
Item Buying Avg Demand Selling Avg Supply Age/Days
---------------------------------------------------------------------------------------------
+METALS
Gold 10,968 9,845 163,205H 2.49
+TEXTILES
Military Grade Fabrics
trade.py market "Facece/Fortress York" -vv
Item Buying Avg Demand Selling Avg Supply Age/Days
-----------------------------------------------------------------------------------------------
+METALS
Gold
+TEXTILES
Military Grade Fabrics 3,557 1,115 123,986M 2.18The route says to pick up Military Grade Fabrics from Brooks Terminal but the market output says that there are none. Likewise for God at Fortress York.
It looks to me as though the route information is the wrong way round and it should be:
Code:
Load from Turir/Brooks Terminal (42ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Metals/Gold 9,304cr vs 10,968cr, 2 days vs 2 days
Expect to gain 1,198,080cr (1,664cr/ton)
Load from Facece/Fortress York (913ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
720 x Textiles/Military Grade Fabrics 625cr vs 3,557cr, 2 days vs 2 days
Expect to gain 2,111,040cr (2,932cr/ton)Or is it just late on a Tuesday evening and the heat is affecting my brain cell?
Last edited:
Mark, first results on TD Helper v2 beta test. I ran through your instructions and downloaded TDH v2, the latest Trade Dangerous from Eyeonus and the latest EDDBlink. Started up the new TD Helper and hit the Update DB button. Practically straight away the display showed "Database is locked. Waiting for access". That line was then repeated every second or so. I left it running for about 10 minutes but looking at my Trade\data folder, the file TradeDangerous.db existed but remained stubbornly at 0 bytes. My brother was doing the same thing and he got the same results.
Obviously at this stage, there was no database to work with but it didn't look like one was being created either. It just kept showing the Database is locked message.
We eventually abandoned that command and ran the eddblink -O clean command to create the database.
Back to TD Helper once the database existed and before clicking anything in the window. the words Building database appeared. Only a few seconds later, everything was ready to go. We both filled in the relevant fields that we wanted and did a 4 hop 2 jump run from our current location. In my case, 10.191 seconds later, the results appeared. Fantastic!
So, in summary, with an existing database it all seems to work very well. Something is not right with the initial database creation at the moment. Perhaps, if we had left it running for the 90 minutes that one of your examples took, it may have completed but was yours displaying anything while it was doing that? All we had was the repeating Database is locked message.
However, with the old version, I always used to get up to 60 seconds of "Not Responding" after making any changes to any of the fields or running a command. I still get them on the new version but only momentary ones - probably 0.25 seconds then off it goes doing what it is supposed to so there is quite a performance improvement there!
I'll let you know if anything else crops up!
Thanks again for doing this.
No:
Code:
if tdenv.maxAge:
maxDays = datetime.timedelta(days=tdenv.maxAge)
cutoff = datetime.datetime.now() - maxDays
wheres.append("(modified >= ?)")
binds.append(str(cutoff.replace(microsecond=0)))With an age of 2, it ignores anything that is more than precisely 48 hours old.
On the other hand, the depiction of a station's age is "rounded" quite a bit:
Code:
def describeAge(ageInSeconds):
"""
Turns an age (in seconds) into a text representation.
"""
hours = int(ageInSeconds / 3600)
if hours < 1:
return "<1 hr"
if hours == 1:
return "1 hr"
if hours < 48:
return str(hours) + " hrs"
days = int(hours / 24)
if days < 90:
return str(days) + " days"
months = int(days / 31)
return str(months) + " mths"Also, I looked at the data for the first station, apparently it's been updated since your post, because the modified time is saying 26 June when I looked at it in the DB, so I have no idea how old it actually was when you were doing this.
Last edited: