In-Development TradeDangerous: power-user trade optimizer

maddavo

maddavo
With the v1.4 update, the Galaxy Map system searching is working again. So tonight I've added around 75 systems to the System.csv . You can import the new System.csv in the usual way with the import plugin option:
trade import -P maddavo -O systems

I'll try to add a chunk of systems closest to SOL every few days.
 

Loriath

Loriath
maddavo said:
With the v1.4 update, the Galaxy Map system searching is working again. So tonight I've added around 75 systems to the System.csv . You can import the new System.csv in the usual way with the import plugin option:
trade import -P maddavo -O systems.
Click to expand...

Awesome. Question about the EDAPI. When it asks for the distance to the station, If I enter anything other than 0 it tosses errors. I have tried to capture them, but since I am pulling the edapi from a batch file called from VA, it goes by to quickly. Is there any secret to how this data should be entered in case I am doing it wrong?

(If I get one tonight I will repull the data in a separate window to capture the error messages)
 

Eggplant!

Eggplant!
Loriath said:
Awesome. Question about the EDAPI. When it asks for the distance to the station, If I enter anything other than 0 it tosses errors. I have tried to capture them, but since I am pulling the edapi from a batch file called from VA, it goes by to quickly. Is there any secret to how this data should be entered in case I am doing it wrong?

(If I get one tonight I will repull the data in a separate window to capture the error messages)
Click to expand...

Hmm.. should just be an integer. What version of the plugin? Error output would be grand.
 

maddavo

maddavo
Loriath said:
Question about the EDAPI. When it asks for the distance to the station...
Click to expand...

Does EDAPI ask for the station Ls distance? I have never seen it do that - for stations where the value in Station.csv is 0Ls I guess? I didn't even know that. What does it do with the distance (I assume it adds to your local Station.csv) - does it upload it anywhere (ie: to Maddavo or EDDB)?
 

CmdrJacks

CmdrJacks
In the version I've been using ('3', '2', '0') (not upgraded yet):
Code: 
lsFromStar = input(
                    "Update distance from star (enter for 0): "
                ) or 0
                lsFromStar = int(lsFromStar)

int(3.2) would round to 3 but because lsFromStar is assigned to the input it is actually performing a int("3.2") which gives an invalid literal. To workaround I just round up the numbers when inputting. An ugly way to fix it in the code is to go to the line:

Code: 
lsFromStar = int(lsFromStar)

and change it to

Code: 
lsFromStar = int(float(lsFromStar))

This should work for all numbers. Text will still crash it though with a ValueError.
 
Last edited:

Minyeni

M
maddavo said:
Aha! Superb.
Click to expand...
In maddavo_plug.py, modify line 695 to read:
Code: 
        tdenv.mergeImport = False if self.getOption("nomerge") else True
then, at the top of the same file, after line 51 (the last current option), add this:
Code: 
        'nomerge': "Don't merge the new .prices data with the old, overwrite it.",
By adding "-O nomerge" to your use of -P maddavo you can overwrite, instead of merging, the new data.
 

gazelle

gazelle
Minyeni said:
In maddavo_plug.py, modify line 695 to read:
Click to expand...

You just need to delete the line, there is allready another option (--merge-import) to switch on merging (see my other post #1944).

edit:

I think the permanent merging in the maddavo plugin was done because of the v1 schema of the EDDN. Now that EDDN switched to the v2 it's not needed anymore, so just deleting the line should be enough.
 
Last edited:

Eggplant!

Eggplant!
maddavo said:
Does EDAPI ask for the station Ls distance? I have never seen it do that - for stations where the value in Station.csv is 0Ls I guess? I didn't even know that. What does it do with the distance (I assume it adds to your local Station.csv) - does it upload it anywhere (ie: to Maddavo or EDDB)?
Click to expand...

If it finds missing info in Station.csv (distance, shipyard, black market, pad size, etc), it will prompt the user to fill it in. Right now, it just adds it to your local Station.csv. As far as I know, EDDN doesn't have a schema for station info, so it's up to the user to manually upload to maddavo.

CmdrJacks said:
(Points out bad int casting.)
Click to expand...

Thanks! I've fixed this in 3.5.2. Please feel free to submit bug reports via bitbucket.

Minyeni said:
Your README says to drop edapi_plug.py into the TD plugins directory; where do the other .py files get dropped?
Click to expand...

The tool also runs stand alone and there is an EDDN client I use for debugging. You can ignore the rest if you are only interested in the TD plugin.

And on that note...

3.5.2 of the plugin in available. It fixes the bug CmdrJacks pointed out with entering Ls data.

https://bitbucket.org/orphu/edapi/src/aa3e25dcd203b7cbbe101bd05b4448755bf9d068?at=3.5.2
 
Last edited:

NE1

NE1
just started getting this error...

Code: 
C:\Program Files Games\Frontier\TradeDangerous\repository>trade import -P maddavo -O corrections,stations,shipvendors,exportcsv,use2d,force
NOTE: Importing Corrections
Traceback (most recent call last):
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 102, in <module>
    main(sys.argv)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 76, in main
    results = cmdenv.run(tdb)

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\import_cmd.py", line 111, in run
    if not plugin.run():

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 707, in run
    self.import_corrections()
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 173, in import_corrections
    for src, oldName, newName in stream:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 132, in csv_stream_rows
    for _, values in transfers.CSVStream(url, tdenv=self.tdenv):

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 291, in __iter__
    for values in csvin:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 270, in next_line
    line = next(self.lines)

Followed by some python lib errors (guesing caused by above)

  File "C:\Python34\lib\site-packages\requests\models.py", line 715, in iter_lines
    for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
  File "C:\Python34\lib\site-packages\requests\models.py", line 673, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 308, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 243, in read
    data = self._fp.read(amt)
  File "C:\Python34\lib\http\client.py", line 500, in read
    return super(HTTPResponse, self).read(amt)
  File "C:\Python34\lib\http\client.py", line 539, in readinto
    n = self.fp.readinto(b)
  File "C:\Python34\lib\socket.py", line 374, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

C:\Program Files Games\Frontier\TradeDangerous\repository>

any help?
 

maddavo

maddavo
NE1 said:
just started getting this error...

Code: 
C:\Program Files Games\Frontier\TradeDangerous\repository>trade import -P maddavo -O corrections,stations,shipvendors,exportcsv,use2d,force
NOTE: Importing Corrections
Traceback (most recent call last):
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 102, in <module>
    main(sys.argv)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 76, in main
    results = cmdenv.run(tdb)

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\import_cmd.py", line 111, in run
    if not plugin.run():

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 707, in run
    self.import_corrections()
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 173, in import_corrections
    for src, oldName, newName in stream:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 132, in csv_stream_rows
    for _, values in transfers.CSVStream(url, tdenv=self.tdenv):

  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 291, in __iter__
    for values in csvin:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 270, in next_line
    line = next(self.lines)

Followed by some python lib errors (guesing caused by above)

  File "C:\Python34\lib\site-packages\requests\models.py", line 715, in iter_lines
    for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
  File "C:\Python34\lib\site-packages\requests\models.py", line 673, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 308, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 243, in read
    data = self._fp.read(amt)
  File "C:\Python34\lib\http\client.py", line 500, in read
    return super(HTTPResponse, self).read(amt)
  File "C:\Python34\lib\http\client.py", line 539, in readinto
    n = self.fp.readinto(b)
  File "C:\Python34\lib\socket.py", line 374, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

C:\Program Files Games\Frontier\TradeDangerous\repository>
any help?
Click to expand...

Just saw ur post. If I'm reading that right it looks like there was some problem downloading the corrections.csv? I have been using the import plugin OK pretty regularly over the past 4 days since ur post. Does the error happen consistently? Are you able to get to the website in a browser? If so, can you access any of the files in a browser that are being downloaded by the plugin?

Cheers,
Maddavo
 

NE1

NE1
maddavo said:
Just saw ur post. If I'm reading that right it looks like there was some problem downloading the corrections.csv? I have been using the import plugin OK pretty regularly over the past 4 days since ur post. Does the error happen consistently? Are you able to get to the website in a browser? If so, can you access any of the files in a browser that are being downloaded by the plugin?

Cheers,
Maddavo
Click to expand...

Yes i can access all the files from the browser; so did a manual d/l of required files and am now getting this error!
when i do an import or buildcache -f

Code: 
C:\Program Files Games\Frontier\TradeDangerous\repository>trade import -P maddavo -O stations,shipvendors,exportcsv,use2d,force
NOTE: Rebuilding cache file: this may take a moment.
*** INTERNAL ERROR: NOT NULL constraint failed: ShipVendor.ship_id
CSV File: D:\Profile\Program Files\FrontierApps\TradeDangerous\repository\data\ShipVendor.csv:5291
SQL Query: INSERT INTO ShipVendor (station_id,ship_id,modified) VALUES((SELECT Station.station_id FROM Station INNER JOIN System USING(system_id) WHERE System.name = ? AND Station.name = ?),(SELECT Ship.ship_id FROM Ship WHERE Ship.name = ?),?)
Params: ['Awawar', 'Cartwright Orbital', 'Federation_Dropship_MkII', '2015-10-26 02:28:06']
 

NE1

NE1
maddavo said:
When you do a buildcache, use the -fi option.
Click to expand...

now getting this error...

trade.py buildcache -fi
Code: 
C:\Program Files Games\Frontier\TradeDangerous\repository>trade.py buildcache -fi
NOTE: Rebuilding cache file: this may take a moment.
*** INTERNAL ERROR: NOT NULL constraint failed: ShipVendor.ship_id
CSV File: D:\Profile\Program Files\FrontierApps\TradeDangerous\repository\data\ShipVendor.csv:5291
SQL Query: INSERT INTO ShipVendor (station_id,ship_id,modified) VALUES((SELECT Station.station_id FROM Station INNER JOIN System USING(system_id) WHERE System.name = ? AND Station.name = ?),(SELECT Ship.ship_id FROM Ship WHERE Ship.name = ?),?)
Params: ['Awawar', 'Cartwright Orbital', 'Federation_Dropship_MkII', '2015-10-26 02:28:06']

if i remove "'Awawar','Cartwright Orbital','Federation_Dropship_MkII','2015-10-26 02:28:06'" from the file i then get the same error
but this time its "'BD+19 2929', 'Wilcutt Enterprise', 'Federation_Dropship_MkII', '2015-10-11 15:24:15'"
--
and this when i use corrections
trade.py import --plug=maddavo --opt=corrections
Code: 
C:\Program Files Games\Frontier\TradeDangerous\repository>trade.py import --plug=maddavo --opt=corrections
NOTE: Importing Corrections
Traceback (most recent call last):
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 102, in <module>
    main(sys.argv)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 76, in main
    results = cmdenv.run(tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\import_cmd.py", line 111, in run
    if not plugin.run():
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 707, in run
    self.import_corrections()
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 173, in import_corrections
    for src, oldName, newName in stream:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\plugins\maddavo_plug.py", line 132, in csv_stream_rows
    for _, values in transfers.CSVStream(url, tdenv=self.tdenv):
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 291, in __iter__
    for values in csvin:
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\transfers.py", line 270, in next_line
    line = next(self.lines)
  File "C:\Python34\lib\site-packages\requests\models.py", line 715, in iter_lines
    for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
  File "C:\Python34\lib\site-packages\requests\models.py", line 673, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 308, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "C:\Python34\lib\site-packages\requests\packages\urllib3\response.py", line 243, in read
    data = self._fp.read(amt)
  File "C:\Python34\lib\http\client.py", line 500, in read
    return super(HTTPResponse, self).read(amt)
  File "C:\Python34\lib\http\client.py", line 539, in readinto
    n = self.fp.readinto(b)
  File "C:\Python34\lib\socket.py", line 374, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
 
Last edited:

gazelle

gazelle
NE1 said:
if i remove "'Awawar','Cartwright Orbital','Federation_Dropship_MkII','2015-10-26 02:28:06'" from the file i then get the same error but this time its "'BD+19 2929', 'Wilcutt Enterprise', 'Federation_Dropship_MkII', '2015-10-11 15:24:15'"
Click to expand...

The Ship "Federation_Dropship_MkII" is wrong. It should be called "Federal Assault Ship". You may add the line
Code: 
'Federation_Dropship_MkII',19814205
to the file data/Ship.csv for a workaround.

edit: Of course it would be better to get the latest "Ship.csv" file from maddavo and update your EDAPI plugin (see post #1973, this might be where this is comming from) and search and replace all "Federation_Dropship_MkII" with "Federal Assault Ship" in the file data/ShipVendor.csv.
 
Last edited:

maddavo

maddavo
OK yeah, I see. There is some API tool that has reported "Federation_Dropship_MkII" instead of "Federal Assault Ship". Normally a plugin import ignores that but a buildcache barfs.

I've fixed up the ShipVendor.csv and updated the translation to eliminate that issue in future.
 

Gimi

Gimi
Any solution to this:
@HIP 20524/Marcy Enterprise
<tradedb.Station object at 0x03589170>
WARNING: import.prices:69 ERROR Unrecognized item name: "HAFNIUM178"
WARNING: import.prices:121 ERROR Unrecognized item name: "MILITARY IN

WARNING: import.prices:123 ERROR Unrecognized item name: "U S S CARGO
TEFACT"
WARNING: import.prices:125 ERROR Unrecognized item name: "U S S CARGO
AL CHEMICALS"
WARNING: import.prices:127 ERROR Unrecognized item name: "U S S CARGO
LANS"
WARNING: import.prices:129 ERROR Unrecognized item name: "U S S CARGO
TECH"
WARNING: import.prices:131 ERROR Unrecognized item name: "U S S CARGO
SMISSIONS"
WARNING: import.prices:133 ERROR Unrecognized item name: "U S S CARGO
BLUEPRINTS"
WARNING: import.prices:135 ERROR Unrecognized item name: "U S S CARGO
"
NOTE: Import complete: 74 updated items over 1 stations in 1 systems
Click to expand...

I'm using TDHelper by the way
 

gazelle

gazelle
Not much info you have given.

Looks like you tried to import some prices with "Salvage" items. These can not be bought and so TD doesn't know them.
 
You must log in or register to reply here.
Back
Top Bottom