In-Development TradeDangerous: power-user trade optimizer

eyeonus · Jun 22, 2018

Tromador said:
I guess you don't remember back to April.
https://forums.frontier.co.uk/showt...de-optimizer?p=6651312&viewfull=1#post6651312
Hence my annoyance.

From memory Python doesn't even demand you install the package unless you add the extra line and once the package is installed, the extra line isn't needed anyway.

I do remember that conversation.

This is not the same.

His issue was that the import needs to be urllib.request, not merely urllib.

His statement, "eddblink asked if I wanted to install the request module", is in error, because EDDBlink never offers to install anything. TD offers to install 'requests', which, again, is a different package.

urllib.request != requests

The only module that TD ever asks if you would like it to install is 'requests'.

Tromador · Jun 22, 2018

That is exactly what I said:

Tromador said:
I had to add @ line 15
import urllib.request

*sigh* Never mind.

eyeonus · Jun 22, 2018

Okay, no, I don't remember seeing that.

I thought you were referring to whether or not 'requests' needed to be installed as part of the installation process, which you also said in the same post (, although you also mistakenly said you needed to install 'request' not 'requests').

Considering this, I think you can see where the confusion comes from:

Tromador said:
I have been saying for ages that pip install requests is a required step in the installation procedure.

You just confirmed the exact step by step that I posted some pages back.

Eyeonus, now do you believe me?

Tromador · Jun 22, 2018

You said

eyeonus said:
I don't think you need to add the import, just the pip install.

Which was the correct answer, but seemed to get lost (possibly because of a typo or mistranscription of requests), so to repeat and summarize what I said & tested at the time (I haven't retested, but anyway).

If you include the extra import, then python automatically begs for the requests module to be installed.
But if you install the module first, then you don't need the extra import.

Hence my suggestion that including the module install in the documentation would be a good idea.

And if we can get confused over it, you can bet anyone will get confused over it - and to avoid further postings from new users about "it asked to install, or I had to install requests"

Can we please - again - put this module into the documentation as a requirement, to *avoid* confusion?

eyeonus · Jun 22, 2018

Tromador said:
You said

Which was the correct answer, but seemed to get lost (possibly because of a typo or mistranscription of requests), so to repeat and summarize what I said & tested at the time (I haven't retested, but anyway).

If you include the extra import, then python automatically begs for the requests module to be installed.
But if you install the module first, then you don't need the extra import.

Hence my suggestion that including the module install in the documentation would be a good idea.

And if we can get confused over it, you can bet anyone will get confused over it - and to avoid further postings from new users about "it asked to install, or I had to install requests"

It's actually not the correct answer. It's a wrong answer that somehow works.

I have absolutely no idea why installing the 'requests' package has the effect of making that error go away, but if you look up the error, you'll see- as I did- that everyone else who had that problem was universally told to explicitly import 'urllib.request'.

The code has been changed to correctly import 'urllib.request', so this will no longer be a problem, and I apologize for taking 3 months to stop being wrong about this.

Can we please - again - put this module into the documentation as a requirement, to *avoid* confusion?

EDDBlink doesn't require the 'requests' package. TD does, and I've decided I'm going to refactor transfers.py to not use 'requests', so no.

Tromador · Jun 22, 2018

eyeonus said:
EDDBlink doesn't require the 'requests' package. TD does, and I've decided I'm going to refactor transfers.py to not use 'requests', so no.

Well, in fairness it's all the same documentation now, so it doesn't really matter which component needs the package. However if you want to go that far to avoid a sentence in an install guide, knock yourself out.

eyeonus · Jun 22, 2018

Tromador said:
Well, in fairness it's all the same documentation now, so it doesn't really matter which component needs the package. However if you want to go that far to avoid a sentence in an install guide, knock yourself out.

I'm not doing it to avoid a sentence. I'm doing it to avoid using third-party software. That's why it isn't included in the Python installation.

Tromador · Jun 22, 2018

eyeonus said:
I'm not doing it to avoid a sentence. I'm doing it to avoid using third-party software. That's why it isn't included in the Python installation.

So are lots of incredibly useful python, perl, $LANGUAGE libraries, even de facto standard ones, third party - doesn't make them a bad thing...

...ok I'll stop yanking your chain now.

Tromador · Jun 22, 2018

I am working on the wiki today - specifically updating the Setup Guide.

Is anyone running on a Mac and can check and comment on that part of the guide? If not, I'll put a note that the instructions are old and unsupported, but leave them in the hopes they are still relevant and useful.

My own knowledge running a Mac basically is click hopefully in the gui, then fail and say "Well, it's BSD isn't it? Gimme a shell prompt and I'll do my best." This is usually followed by me finding it's unlike any BSD I ever used and washing my hands of the problem.

MarkAusten · Jun 22, 2018

Tromador said:
I am working on the wiki today - specifically updating the Setup Guide.

Is anyone running on a Mac and can check and comment on that part of the guide? If not, I'll put a note that the instructions are old and unsupported, but leave them in the hopes they are still relevant and useful.

My own knowledge running a Mac basically is click hopefully in the gui, then fail and say "Well, it's BSD isn't it? Gimme a shell prompt and I'll do my best." This is usually followed by me finding it's unlike any BSD I ever used and washing my hands of the problem.

Which Wiki would that be?

Tromador · Jun 22, 2018

MarkAusten said:
Which Wiki would that be?

https://github.com/eyeonus/Trade-Dangerous/wiki

Which is (other than the Setup Guide I've been hacking this afternoon) a clone of Oliver's wiki.

CMDR Ibizan · Jun 22, 2018

Tromador said:
https://github.com/eyeonus/Trade-Dangerous/wiki

Which is (other than the Setup Guide I've been hacking this afternoon) a clone of Oliver's wiki.

Tromador,

Have you seen Bgol's page?

https://bitbucket.org/bgol/tradedangerous

There are absolutely loads of instructions for TD on that page that you might be able to use. It will probably save you some typing....

MarkAusten · Jun 22, 2018

Tromador said:
https://github.com/eyeonus/Trade-Dangerous/wiki

Which is (other than the Setup Guide I've been hacking this afternoon) a clone of Oliver's wiki.

Home page
Missing the abbreviated form on the home page below the first example.
What do line 1 - line 5 refer to? I suspect that the results have been omitted.

Setup Page Mac Users:

I suggest that you add instructions to download the latest python version from https://www.python.org/downloads/mac-osx/ and install from that for users who do not have home-brew installed.

Instruction line 3 should be on a new line.

You really think that adding virtualenv is easier than typing python3?

Virtualenv did not work on my Mac and may not work on other Macs either. Probably best to stick to using "python3 trade.py ..." so I suggest that you put the virtualenv thing into a "Hints & Tips" or "FAQs" section and refer to it in the setup otherwise it does look like you are saying that this is the correct way to run TradeDangerous.

Tromador · Jun 22, 2018

MarkAusten said:
Home page
Missing the abbreviated form on the home page below the first example.
What do line 1 - line 5 refer to? I suspect that the results have been omitted.

It's there, but it's not in .md format, I need to convert it and not gotten around to it yet.

Setup Page Mac Users:

I suggest that you add instructions to download the latest python version from https://www.python.org/downloads/mac-osx/ and install from that for users who do not have home-brew installed.

Instruction line 3 should be on a new line.

You really think that adding virtualenv is easier than typing python3?

Virtualenv did not work on my Mac and may not work on other Macs either. Probably best to stick to using "python3 trade.py ..." so I suggest that you put the virtualenv thing into a "Hints & Tips" or "FAQs" section and refer to it in the setup otherwise it does look like you are saying that this is the correct way to run TradeDangerous.

I have no idea about virtualenv - these were Oliver's original instructions and again to stress - I know next to nothing about MacOs, or I would rewrite these instructions myself. If you can write up a correct and current set of MacOS instructions I would be very glad to include them.

Tromador · Jun 22, 2018

CMDR Ibizan said:
Tromador,

Have you seen Bgol's page?

https://bitbucket.org/bgol/tradedangerous

There are absolutely loads of instructions for TD on that page that you might be able to use. It will probably save you some typing....

All of which were included in the main readme when Eyeonus forked the repository (and are still on the main page @ https://github.com/eyeonus/Trade-Dangerous - just that bitbucket doesn't hide the readme underneath the list of source files).

Tromador · Jun 22, 2018

Are we sure about that new ls-per curve? It just tried to have me run hydrogen (fair, the station really only had that or biowaste) to a place the better part of 200K ls from the star.

Additionally, no such place as Daruwutja/Acropolis - not sure where that comes from, though this must be a separate issue.
I guess if I had remembered to include an age term, neither of these would have popped up, but I did forget and so here they are.

Problem 1 - Please don't send me to Amikul/Benz with hydrogen. The documentation states that there is a default ls penalty of 0.5, is that actually the case?
Problem 2 - Stale data about a place which doesn't exist (I presume it must have at one time).

Antiproblem 1 - It really is running a LOT quicker with the new "fill the hold" algorithm. Good job guys.

trade.py run --from paesan/abel --to cubeo --hops 10 -v --pla=YN? --credits=35000000 --capacity=150 --ly-per=27.98 --empty-ly=32.03 --insurance=15500000 --unique --progress --pad-size=L

Code:

* Hop   1: .........1 origins
* Hop   2: .......109 origins
* Hop   3: .....1,542 origins
NOTE: Pruned 1 origins too far from any end stations
* Hop   4: .....5,056 origins
NOTE: Pruned 767 origins too far from any end stations
* Hop   5: ....10,361 origins
NOTE: Pruned 4428 origins too far from any end stations
* Hop   6: ....13,562 origins
NOTE: Pruned 9924 origins too far from any end stations
* Hop   7: ....12,174 origins
NOTE: Pruned 12099 origins too far from any end stations
* Hop   8: .....7,262 origins
NOTE: Pruned 9599 origins too far from any end stations
* Hop   9: .....3,165 origins
NOTE: Pruned 5943 origins too far from any end stations
* Hop  10: .......652 origins
Paesan/Abel Landing -> Cubeo/Medupe City (score: 4252233.733499)
  Load from Paesan/Abel Landing: 150 x Chemicals/Hydrogen Fuel (@84cr),
  Dock at Ambikul/Benz Settlement
  Load from Ambikul/Benz Settlement: 150 x Medicines/Basic Medicines (@359cr),
  Dock at Daruwutja/Acropolis
  Load from Daruwutja/Acropolis: 145 x Weapons/Reactive Armour (@227cr), 5 x Weapons/Personal Weapons (@416cr),
  Dock at 64 Piscium/Miller's Inheritance
  Load from 64 Piscium/Miller's Inheritance: 150 x Metals/Thallium (@2126cr),
  Dock at LFT 174/Berezovoy Gateway
  Load from LFT 174/Berezovoy Gateway: 150 x Medicines/Basic Medicines (@343cr),
  Dock at Mari/Guidoni Hub
  Load from Mari/Guidoni Hub: 150 x Foods/Tea (@1375cr),
  Dock at Gl 606.1 B/Harvey Ring
  Load from Gl 606.1 B/Harvey Ring: 149 x Slavery/Imperial Slaves (@14230cr), 1 x Machinery/Building Fabricators (@1272cr),
  Dock at Kipgeninka/Jordan Enterprise
  Load from Kipgeninka/Jordan Enterprise: 150 x Metals/Tantalum (@2245cr),
  Dock at Kamocan/Gasparis Vision
  Load from Kamocan/Gasparis Vision: 150 x Slavery/Imperial Slaves (@14196cr),
  Dock at Djakah/Al-Khujandi Hub
  Load from Djakah/Al-Khujandi Hub: 91 x Metals/Gold (@9881cr), 59 x Metals/Palladium (@12814cr),
  Dock at Cubeo/Medupe City
  Finish Cubeo/Medupe City + 4,249,183cr (2,832cr/ton)=> 39,249,183cr

gazelle · Jun 22, 2018

eyeonus said:
Solution to problem of commodities that have been removed as a saleable item from a station not being updated as such and remaining in DB with the old data, because both EDDN and EDDB updates do not include removed items

Why don't you just DELETE all items from the station and then INSERT all items from the message? This would also get rid of the INSERT or "UPDATE if fail" you're doing.

edit: BTW, thanks for continuing the TD support.

eyeonus · Jun 22, 2018

Tromador said:
So are lots of incredibly useful python, perl, $LANGUAGE libraries, even de facto standard ones, third party - doesn't make them a bad thing...

...ok I'll stop yanking your chain now.

I just the don't see the point of making users download a third party package when there's a perfectly suitable one that comes with the language and only one small module uses it.

Tromador · Jun 22, 2018

eyeonus said:
I just the don't see the point of making users download a third party package when there's a perfectly suitable one that comes with the language and only one small module uses it.

Mate... I even told you I was yanking your chain. Drop it

eyeonus · Jun 22, 2018

gazelle said:
Why don't you just DELETE all items from the station and then INSERT all items from the message? This would also get rid of the INSERT or "UPDATE if fail" you're doing.

edit: BTW, thanks for continuing the TD support.

Because DELETE is horribly slow in SQLite, and the listener needs to be as fast as possible, so simply UPDATEing those to 0-out entries allows us to off-load the actual removal to TD, which doesn't need to be super-quick.

(Although TD doesn't even delete, as far as I can tell- when it rebuilds the database, it simply doesn't copy over the entries to be deleted.)

'INSERT, if fail UPDATE' is literally the fastest I can make it. It's even faster than 'SELECT entry, UPDATE if entry, INSERT if no entry', because it's one less SQL call.

Tromador said:
Mate... I even told you I was yanking your chain. Drop it

Nevar!