In-Development TradeDangerous: power-user trade optimizer

OggVorbis · Jul 25, 2015

Just so you guys know, the EDAPI plugin is completely broken now. Instead of sending market reports like this:

Code:

    Item                    Buying     Avg      Demand Selling     Avg     Supply Age/Days
------------------------------------------------------------------------------------------
+CHEMICALS
    Hydrogen Fuel                                          115     102   928,545M         
+CONSUMER ITEMS
    Clothing                                               238     239   329,198M         
    Consumer Technology      7,510   7,258    136,634H                                0.00
    Domestic Appliances                                    438     437   195,925M         
+FOODS
    Algae                      257     178  3,369,268H                                0.00
    Animal Meat              1,635   1,457    272,184H                                0.00
    Coffee                   1,543   1,434     54,857M                                0.00
    Fish                       582     492    754,443H                                0.00
    Food Cartridges                                         45      49   288,101H         
    Fruit And Vegetables       433     377    197,990M                                0.00
    Grain                      334     259  1,561,655H                                0.00
    Synthetic Meat             363     342    131,614M                                0.00
    Tea                      1,811   1,639    235,374H                                0.00
+INDUSTRIAL MATERIALS
    Polymers                   277     164  9,639,970H                                0.00
    Semiconductors           1,169     963  6,592,070H                                0.00
    Superconductors          7,505   6,885    674,041H                                0.00
+LEGAL DRUGS
    Beer                       303     235  1,050,945H                                0.00
    Liquor                                                 536     543       334M         
    Narcotics                9,210   9,566        145L                                0.00
    Tobacco                  5,326   5,128     56,479M                                0.00
    Wine                       396     324  1,264,540H                                0.00
+MACHINERY
    Atmospheric Processors                                 287     305   326,181H         
    Crop Harvesters                                      1,863   1,969   257,662H         
    Marine Equipment                                     3,752   3,891 1,883,882H         
    Mineral Extractors                                     429     469   936,485H         
    Power Generators                                       374     412 5,127,986H         
    Water Purifiers                                        191     210   137,715H         
+MEDICINES
    Basic Medicines                                        238     229    32,733M         
    Combat Stabilisers       3,100   3,141      1,935M                                0.00
    Performance Enhancers    7,510   7,323    189,769H                                0.00
    Progenitor Cells         7,481   7,230     74,244H                                0.00
+METALS
    Aluminium                  486     338  3,186,007H                                0.00
    Beryllium                9,080   8,468    286,538H                                0.00
    Cobalt                     852     682  1,808,853H                                0.00
    Copper                     656     469  5,489,619H                                0.00
    Gallium                  5,830   5,350  2,461,232H                                0.00
    Gold                     9,300   9,663  2,415,542H                                0.00
    Indium                   6,592   6,002    366,030H                                0.00
    Lithium                  1,944   1,672  2,063,092H                                0.00
    Osmium                   7,510   7,518  1,339,245H                                0.00
    Palladium               12,832  13,734  1,870,259H                                0.00
    Platinum                19,715  19,750     59,744H                                0.00
    Silver                   4,929   4,950    603,147H                                0.00
    Tantalum                 4,537   4,118  3,016,954H                                0.00
    Titanium                 1,248   1,016  1,020,337M                                0.00
    Uranium                  3,137   2,808  4,116,971H                                0.00
+MINERALS
    Painite                 36,044  35,688     28,115H                                0.00
+SALVAGE
    SAP 8 Core Container    60,073  60,177    562,278H                                0.00
+SLAVERY
    Imperial Slaves                                     13,641  15,969   195,714M         
+TECHNOLOGY
    Auto-Fabricators         3,617   4,006     53,858L                                0.00
    Computer Components                                    437     398    19,640M         
    H.E. Suits                 425     401  2,044,119H                                0.00
    Robotics                 2,181   2,011    556,829H                                0.00
+TEXTILES
    Leather                    305     251 18,906,031H                                0.00
    Natural Fabrics            584     514 13,556,571H                                0.00
    Synthetic Fabrics          313     202  3,328,055H                                0.00
+WASTE
    Biowaste                                                20      18    37,877M         
    Scrap                                                   31      33   741,618H         
+WEAPONS
    Battle Weapons                                       6,556   6,527     4,676M         
    Non-Lethal Weapons       2,024   2,016      3,008M                                0.00
    Personal Weapons                                     4,031   4,036       375M         
    Reactive Armour          2,407   2,300     10,412M                                0.00

...it's sending reports like this:

Code:

    Item                    Buying     Avg Demand Selling     Avg Supply Age/Days
---------------------------------------------------------------------------------
+CHEMICALS
    Hydrogen Fuel                                     115     102      -         
+CONSUMER ITEMS
    Clothing                                          238     239      -         
    Domestic Appliances                               438     437      -         
+FOODS
    Food Cartridges                                    45      49      -         
+LEGAL DRUGS
    Liquor                                            536     543      -         
+MACHINERY
    Atmospheric Processors                            287     305      -         
    Crop Harvesters                                 1,863   1,969      -         
    Marine Equipment                                3,752   3,891      -         
    Mineral Extractors                                429     469      -         
    Power Generators                                  374     412      -         
    Water Purifiers                                   191     210      -         
+MEDICINES
    Basic Medicines                                   238     229      -         
+SLAVERY
    Imperial Slaves                                13,641  15,969      -         
+TECHNOLOGY
    Computer Components                               437     398      -         
+WASTE
    Biowaste                                           20      18      -         
    Scrap                                              31      33      -         
+WEAPONS
    Battle Weapons                                  6,556   6,527      -         
    Personal Weapons                                4,031   4,036      -

It's very frustrating to know that all my reports over the last four days were useless. Click here to see the change to undo.

DRY411S · Jul 26, 2015

I add systems using EDSC, not the TD plug-in. I just find it easier to do it that way. I then add the systems to system.csv.

When do systems that I add to EDSC make it into maddavo's csv file?

CmdrJacks · Jul 26, 2015

OggVorbis said:
It's very frustrating to know that all my reports over the last four days were useless. Click here to see the change to undo.

That code isn't right, it's skipping stock. It assigns '-' if there is stock at the station. This works:

Code:

                f.write("\t+ {}\n".format(commodity['categoryname']))

                try:
                    commodity['stock'] = int(commodity['stock'])
                except ValueError:
                    commodity['stock'] = int(0)

                try:
                    commodity['demand'] = int(commodity['demand'])
                except ValueError:
                    commodity['demand'] = int(0)

                try:
                    commodity['demandBracket'] = int(commodity['demandBracket'])
                except ValueError:
                    commodity['demandBracket'] = int(0)


                # If stock is zero, list it as unavailable.
                if commodity['stock'] == 0:
                    commodity['stock'] = '-'
                else:
                    demand = bracket_levels[int(commodity['stockBracket'])]
                    commodity['stock'] = str(int(commodity['stock']))+demand

                # If demand is zero, zero out the sell price.
                if (commodity['demand'] == 0 or
                    commodity['demandBracket'] == 0
                   ):
                    commodity['demand'] = '?'
                    commodity['sellPrice'] = 0
                else:
                    demand = bracket_levels[int(commodity['demandBracket'])]
                    commodity['demand'] = str(int(commodity['demand']))+demand

diff --git a/plugins/edapi_plug.py b/plugins/edapi_plug.py
index d82091e..56b7cbb 100644
--- a/plugins/edapi_plug.py
+++ b/plugins/edapi_plug.py
@@ -600,11 +600,28 @@ class ImportPlugin(plugins.ImportPluginBase):
"demandLevel": EDDN._levels[int(commodity['demandBracket'])]
}
)
-
+ print (eddn_market)
+ print(commodity)
f.write("\t+ {}\n".format(commodity['categoryname']))

+ try:
+ commodity['stock'] = int(commodity['stock'])
+ except ValueError:
+ commodity['stock'] = int(0)
+
+ try:
+ commodity['demand'] = int(commodity['demand'])
+ except ValueError:
+ commodity['demand'] = int(0)
+
+ try:
+ commodity['demandBracket'] = int(commodity['demandBracket'])
+ except ValueError:
+ commodity['demandBracket'] = int(0)
+
+
# If stock is zero, list it as unavailable.
- if commodity['stock'] or 0 == 0:
+ if commodity['stock'] == 0:
commodity['stock'] = '-'
else:
demand = bracket_levels[int(commodity['stockBracket'])]
@@ -612,7 +629,7 @@ class ImportPlugin(plugins.ImportPluginBase):

# If demand is zero, zero out the sell price.
if (commodity['demand'] == 0 or
- commodity['demandBracket'] or 0 == 0
+ commodity['demandBracket'] == 0
):
commodity['demand'] = '?'
commodity['sellPrice'] = 0

Not elegant, but it works, at least until it gets fixed upstream.

Norin-Rad · Jul 27, 2015

Hello,

I got here thanks to this discution: https://forums.frontier.co.uk/showthread.php?t=103162
I just want to know if the problem driver to operate the X52 Pro MFD was solved.
I would like to use the MFD pro x52 Elite Dangerous.

I am French and I use google translate, I apologize in advance if my English is not very good.

s0litaire · Jul 28, 2015

is it possible to exclude a commodity from the search?

at the moment every run wants me to load up on Narcotics which are prohibited in my current area and the stations stopped selling them a few months ago.
I'm updating EDDN at every station i'm flying through but it's going to take a while to purge the old Narcotic prices out of the system.
Untill then i'm not going to get any great rads runs using the program.

maddavo · Jul 28, 2015

s0litaire said:
is it possible to exclude a commodity from the search?

at the moment every run wants me to load up on Narcotics which are prohibited in my current area and the stations stopped selling them a few months ago.
I'm updating EDDN at every station i'm flying through but it's going to take a while to purge the old Narcotic prices out of the system.
Untill then i'm not going to get any great rads runs using the program.

use the "-avoid narcotics" option

s0litaire · Jul 28, 2015

maddavo said:
use the "-avoid narcotics" option

thanks ^__^

maddavo · Jul 28, 2015

DRY411S said:
I add systems using EDSC, not the TD plug-in. I just find it easier to do it that way. I then add the systems to system.csv.

When do systems that I add to EDSC make it into maddavo's csv file?

System.csv gets updated when I sync with the repo to get any systems added there, OR also when I add them myself from EDSC using TD's edscupdate .

I have not done this in a LONG time because:
a) There was a sudden massive increase in EDSC systems a while ago and I couldn't keep up (each system is manually verified with one distance when inserting into System.csv)
b) We have all the systems with stations - which is the most important thing.
c) The GM can plot routes now so TD isn't really used for navigation - but understand it can affect trade route generation.

As mentioned, the most important thing was to have all the systems with stations. The next most important thing is to have all the systems in-between systems with stations so trade routes can be considered for a run. As EDSC is capturing all star mappings (which is great - but unnecessary for TD), it's probably worthwhile editing edscupdate so that we just grab the systems within the inhabited sphere. I guess I should flag that on bitbucket.

CmdrJacks · Jul 28, 2015

There is another issue with the edapi plugin. If demand is zero (as is the case with the CGs at the moment) then the buy price is ignored. I have a fix for this and this diff also contains the fix for the double negative conditionals breaking the edapi:

Code:

diff --git a/plugins/edapi_plug.py b/plugins/edapi_plug.py
index d82091e..74699a9 100644
--- a/plugins/edapi_plug.py
+++ b/plugins/edapi_plug.py
@@ -603,8 +603,24 @@ class ImportPlugin(plugins.ImportPluginBase):

                 f.write("\t+ {}\n".format(commodity['categoryname']))

+                try:
+                    commodity['stock'] = int(commodity['stock'])
+                except ValueError:
+                    commodity['stock'] = int(0)
+
+                try:
+                    commodity['demand'] = int(commodity['demand'])
+                except ValueError:
+                    commodity['demand'] = int(0)
+
+                try:
+                    commodity['demandBracket'] = int(commodity['demandBracket'])
+                except ValueError:
+                    commodity['demandBracket'] = int(0)
+
+
                 # If stock is zero, list it as unavailable.
-                if commodity['stock'] or 0 == 0:
+                if commodity['stock'] == 0:
                     commodity['stock'] = '-'
                 else:
                     demand = bracket_levels[int(commodity['stockBracket'])]
@@ -612,7 +628,7 @@ class ImportPlugin(plugins.ImportPluginBase):

                 # If demand is zero, zero out the sell price.
                 if (commodity['demand'] == 0 or
-                    commodity['demandBracket'] or 0 == 0
+                    commodity['demandBracket'] == 0
                    ):
                     commodity['demand'] = '?'
                     commodity['sellPrice'] = 0

OggVorbis · Jul 28, 2015

CmdrJacks said:
There is another issue with the edapi plugin. If demand is zero (as is the case with the CGs at the moment) then the buy price is ignored. I have a fix for this and this diff also contains the fix for the double negative conditionals breaking the edapi:

I'm no programmer. How do I apply this fix?

CmdrJacks · Jul 28, 2015

You can use git to do it or there are tools out there for diff and patch. You want to apply a patch. The simplest way would be to use the patch command. Save the diff as a txt file and:

Code:

patch -p1 < patchfile.txt

This is probably gobbldeygook to you so here is the full file edapi_plug.py which exists in the plugins folder.

Code:

# ----------------------------------------------------------------
# Import plugin that downloads market and ship vendor data from the
# Elite Dangerous mobile API.
# ----------------------------------------------------------------

import cache
import csvexport
from datetime import datetime, timezone
import getpass
import hashlib
import json
import os
import pathlib
import plugins
import pickle
import random
import requests
from requests.utils import dict_from_cookiejar
from requests.utils import cookiejar_from_dict
import sys
import textwrap

__version_info__ = ('3', '2', '0')
__version__ = '.'.join(__version_info__)

# ----------------------------------------------------------------
# Deal with some differences in names between TD, ED and the API.
# ----------------------------------------------------------------

bracket_levels = ('-', 'L', 'M', 'H')

# This translates what the API calls a ship into what TD calls a
# ship.

ship_names = {
    'Adder': 'Adder',
    'Anaconda': 'Anaconda',
    'Asp': 'Asp',
    'CobraMkIII': 'Cobra',
    'DiamondBack': 'Diamondback Scout',
    'DiamondBackXL': 'Diamondback Explorer',
    'Eagle': 'Eagle',
    'Empire_Courier': 'Imperial Courier',
    'Empire_Fighter': 'Empire_Fighter',
    'Empire_Trader': 'Clipper',
    'Federation_Dropship': 'Dropship',
    'Federation_Fighter': 'Federation_Fighter',
    'FerDeLance': 'Fer-de-Lance',
    'Hauler': 'Hauler',
    'Orca': 'Orca',
    'Python': 'Python',
    'SideWinder': 'Sidewinder',
    'Type6': 'Type 6',
    'Type7': 'Type 7',
    'Type9': 'Type 9',
    'Viper': 'Viper',
    'Vulture': 'Vulture',
}

# Categories to ignore. Drones end up here. No idea what they are.
cat_ignore = [
    'NonMarketable',
]

# TD has different names for these.
cat_correct = {
    'Narcotics': 'Legal Drugs',
    'Slaves': 'Slavery',
}

# TD has different names for these.
comm_correct = {
    'Agricultural Medicines': 'Agri-Medicines',
    'Atmospheric Extractors': 'Atmospheric Processors',
    'Auto Fabricators': 'Auto-Fabricators',
    'Basic Narcotics': 'Narcotics',
    'Bio Reducing Lichen': 'Bioreducing Lichen',
    'Hazardous Environment Suits': 'H.E. Suits',
    'Heliostatic Furnaces': 'Microbial Furnaces',
    'Marine Supplies': 'Marine Equipment',
    'Non Lethal Weapons': 'Non-Lethal Weapons',
    'S A P8 Core Container': 'SAP 8 Core Container',
    'Terrain Enrichment Systems': 'Land Enrichment Systems',
}


class EDAPI:
    '''
    A class that handles the Frontier ED API.
    '''

     _agent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X)  AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12B411'  # NOQA
    _baseurl = 'https://companion.orerve.net/'
    _basename = 'edapi'
    _cookiefile = _basename + '.cookies'
    _envfile = _basename + '.vars'

    def __init__(self, basename='edapi', debug=False, cookiefile=None):
        '''
        Initialize
        '''

        # Build common file names from basename.
        self._basename = basename
        if cookiefile:
            self._cookiefile = cookiefile
        else:
            self._cookiefile = self._basename + '.cookies'

        self._envfile = self._basename + '.vars'

        self.debug = debug
        # if self.debug:
        #     import http.client
        #     http.client.HTTPConnection.debuglevel = 3

        # Setup the HTTP session.
        self.opener = requests.Session()

        self.opener.headers = {
            'User-Agent': self._agent
        }

        # Read/create the cookie jar.
        if os.path.exists(self._cookiefile):
            try:
                with open(self._cookiefile, 'rb') as h:
                    self.opener.cookies = cookiejar_from_dict(pickle.load(h))
            except:
                print('Unable to read cookie file.')

        else:
            with open(self._cookiefile, 'wb') as h:
                pickle.dump(dict_from_cookiejar(self.opener.cookies), h)

        # Grab the commander profile
        response = self._getURI('profile')
        try:
            self.profile = response.json()
        except:
            sys.exit('Unable to parse JSON response for /profile!\
                     Try with --debug and report this.')

    def _getBasicURI(self, uri, values=None):
        '''
        Perform a GET/POST to a URI
        '''

        # POST if data is present, otherwise GET.
        if values is None:
            if self.debug:
                print('GET on: ', self._baseurl+uri)
                print(dict_from_cookiejar(self.opener.cookies))
            response = self.opener.get(self._baseurl+uri)
        else:
            if self.debug:
                print('POST on: ', self._baseurl+uri)
                print(dict_from_cookiejar(self.opener.cookies))
            response = self.opener.post(self._baseurl+uri, data=values)

        if self.debug:
            print('Final URL:', response.url)
            print(dict_from_cookiejar(self.opener.cookies))

        # Save the cookies.
        with open(self._cookiefile, 'wb') as h:
            pickle.dump(dict_from_cookiejar(self.opener.cookies), h)

        # Return the response object.
        return response

    def _getURI(self, uri, values=None):
        '''
        Perform a GET/POST and try to login if needed.
        '''

        # Try the URI. If our credentials are no good, try to
        # login then ask again.
        response = self._getBasicURI(uri, values=values)

        if str(response.url).endswith('user/login'):
            self._doLogin()
            response = self._getBasicURI(uri, values=values)

        if str(response.url).endswith('user/login'):
            sys.exit(textwrap.fill(textwrap.dedent("""\
                Something went terribly wrong. The login credentials
                appear correct, but we are being denied access. Sometimes the
                API is slow to update, so if you are authenticating for the
                first time, wait a minute or so and try again. If this
                persists try using --debug and report this.
                """)))

        return response

    def _doLogin(self):
        '''
        Go though the login process
        '''
        # First hit the login page to get our auth cookies set.
        response = self._getBasicURI('')

        # Our current cookies look okay? No need to login.
        if str(response.url).endswith('/'):
            return

        # Perform the login POST.
        print(textwrap.fill(textwrap.dedent("""\
              You do not appear to have any valid login cookies set.
              We will attempt to log you in with your Frontier
              account, and cache your auth cookies for future use.
              THIS WILL NOT STORE YOUR USER NAME AND PASSWORD.
              """)))

        print("\nYour auth cookies will be stored here:")

        print("\n"+self._cookiefile+"\n")

        print(textwrap.fill(textwrap.dedent("""\
            It is advisable that you keep this file secret. It may
            be possible to hijack your account with the information
            it contains.
            """)))

        print(
            "\nIf you are not comfortable with this, "
            "DO NOT USE THIS TOOL."
        )
        print()

        values = {}
        values['email'] = input("User Name (email):")
        values['password'] = getpass.getpass()
        response = self._getBasicURI('user/login', values=values)

        # If we end up being redirected back to login,
        # the login failed.
        if str(response.url).endswith('user/login'):
            sys.exit('Login failed.')

        # Check to see if we need to do the auth token dance.
        if str(response.url).endswith('user/confirm'):
            print()
            print("A verification code should have been sent to your "
                  "email address.")
            print("Please provide that code (case sensitive!)")
            values = {}
            values['code'] = input("Code:")
            response = self._getBasicURI('user/confirm', values=values)


class EDDN:
    _gateways = (
        'http://eddn-gateway.elite-markets.net:8080/upload/',
        # 'http://eddn-gateway.ed-td.space:8080/upload/',
    )

    _schemas = {
        'production': 'http://schemas.elite-markets.net/eddn/commodity/2',
        'test': 'http://schemas.elite-markets.net/eddn/commodity/2/test',
    }

    _debug = True

    # As of 1.3, ED reports four levels.
    _levels = (
        'Low',
        'Low',
        'Med',
        'High',
    )

    def __init__(
        self,
        uploaderID,
        softwareName,
        softwareVersion
    ):
        # Obfuscate uploaderID
        self.uploaderID = hashlib.sha1(uploaderID.encode('utf-8')).hexdigest()
        self.softwareName = softwareName
        self.softwareVersion = softwareVersion

    def publishCommodities(
        self,
        systemName,
        stationName,
        commodities,
        timestamp=0
    ):
        message = {}

        message['$schemaRef'] = self._schemas[('test' if self._debug else 'production')]  # NOQA

        message['header'] = {
            'uploaderID': self.uploaderID,
            'softwareName': self.softwareName,
            'softwareVersion': self.softwareVersion
        }

        if timestamp:
            timestamp = datetime.fromtimestamp(timestamp).isoformat()
        else:
            timestamp = datetime.now(timezone.utc).astimezone().isoformat()

        message['message'] = {
            'systemName': systemName,
            'stationName': stationName,
            'timestamp': timestamp,
            'commodities': commodities,
        }

        url = random.choice(self._gateways)

        headers = {
            'content-type': 'application/json; charset=utf8'
        }

        if self._debug:
            print(
                json.dumps(
                    message,
                    sort_keys=True,
                    indent=4
                )
            )

        r = requests.post(
            url,
            headers=headers,
            data=json.dumps(
                message,
                ensure_ascii=False
            ).encode('utf8'),
            verify=True
        )

        r.raise_for_status()


class ImportPlugin(plugins.ImportPluginBase):
    """
    Plugin that downloads market and ship vendor data from the Elite Dangerous
    mobile API.
    """

    pluginOptions = {
        'eddn': 'Post market prices to EDDN.',
    }

    cookieFile = "edapi.cookies"

    def __init__(self, tdb, tdenv):
        super().__init__(tdb, tdenv)

        self.filename = self.defaultImportFile
        cookieFilePath = pathlib.Path(ImportPlugin.cookieFile)
        self.cookiePath = tdb.dataPath / cookieFilePath

    def run(self):
        tdb, tdenv = self.tdb, self.tdenv

        # Connect to the API, authenticate, and pull down the commander
        # /profile.
        api = EDAPI(cookiefile=str(self.cookiePath))

        # Sanity check that the commander is docked. Otherwise we will get a
        # mismatch between the last system and last station.
        if not api.profile['commander']['docked']:
            print('Commander not docked. Aborting!')
            return False

        # Figure out where we are.
        system = api.profile['lastSystem']['name']
        station = api.profile['lastStarport']['name']
        place = '@{}/{}'.format(system.upper(), station)
        print(place)

        # Reload the cache.
        tdenv.DEBUG0("Checking the cache")
        tdb.close()
        tdb.reloadCache()
        tdb.load(
            maxSystemLinkLy=tdenv.maxSystemLinkLy,
        )
        tdb.close()

        # Check to see if this system is in the Stations file
        try:
            station_lookup = tdb.lookupPlace(place)
        except LookupError:
            station_lookup = None

        print(station_lookup)

        # The station isn't known. Add it.
        if not station_lookup:
            print('Station unknown.')
            print('Adding:', place)
            lsFromStar = input(
                "Distance from star (enter for 0): "
            ) or 0
            lsFromStar = int(lsFromStar)
            blackMarket = input(
                "Black market present (Y, N or enter for ?): "
            ) or '?'
            maxPadSize = input(
                "Max pad size (S, M, L or enter for ?): "
            ) or '?'
            outfitting = input(
                "Outfitting present (Y, N or enter for ?): "
            ) or '?'
            rearm = input(
                "Rearm present (Y, N or enter for ?): "
            ) or '?'
            refuel = input(
                "Refuel present (Y, N or enter for ?): "
            ) or '?'
            repair = input(
                "Repair present (Y, N or enter for ?): "
            ) or '?'
            # This is unreliable, so default to unknown.
            if 'commodities' in api.profile['lastStarport']:
                market = 'Y'
            else:
                market = '?'
            # This is also unreliable, so default to unknown.
            if 'ships' in api.profile['lastStarport']:
                shipyard = 'Y'
            else:
                shipyard = '?'
            system_lookup = tdb.lookupSystem(system)
            if tdb.addLocalStation(
                system=system_lookup,
                name=station,
                lsFromStar=lsFromStar,
                blackMarket=blackMarket,
                maxPadSize=maxPadSize,
                market=market,
                shipyard=shipyard,
                outfitting=outfitting,
                rearm=rearm,
                refuel=refuel,
                repair=repair
            ):
                lines, csvPath = csvexport.exportTableToFile(
                    tdb,
                    tdenv,
                    "Station"
                )
                tdenv.NOTE("{} updated.", csvPath)
                station_lookup = tdb.lookupPlace(place)
            station_lookup = tdb.lookupStation(station, system)
        else:
            # See if we need to update the info for this station.
            lsFromStar = station_lookup.lsFromStar
            blackMarket = station_lookup.blackMarket
            maxPadSize = station_lookup.maxPadSize
            market = station_lookup.market
            shipyard = station_lookup.shipyard
            outfitting = station_lookup.outfitting
            rearm = station_lookup.rearm
            refuel = station_lookup.refuel
            repair = station_lookup.repair

            if lsFromStar == 0:
                lsFromStar = input(
                    "Update distance from star (enter for 0): "
                ) or 0
                lsFromStar = int(lsFromStar)

            if blackMarket is '?':
                blackMarket = input(
                    "Update black market present (Y, N or enter for ?): "
                ) or '?'

            if maxPadSize is '?':
                maxPadSize = input(
                    "Update max pad size (S, M, L or enter for ?): "
                ) or '?'

            if outfitting is '?':
                outfitting = input(
                    "Update outfitting present (Y, N or enter for ?): "
                ) or '?'

            if rearm is '?':
                rearm = input(
                    "Update rearm present (Y, N or enter for ?): "
                ) or '?'

            if refuel is '?':
                refuel = input(
                    "Update refuel present (Y, N or enter for ?): "
                ) or '?'

            if repair is '?':
                repair = input(
                    "Update repair present (Y, N or enter for ?): "
                ) or '?'

            # This is unreliable, so default to unchanged.
            if 'commodities' in api.profile['lastStarport']:
                market = 'Y'

            # This is also unreliable, so default to unchanged.
            if 'ships' in api.profile['lastStarport']:
                shipyard = 'Y'

            if (
                lsFromStar != station_lookup.lsFromStar or
                blackMarket != station_lookup.blackMarket or
                maxPadSize != station_lookup.maxPadSize or
                market != station_lookup.market or
                shipyard != station_lookup.shipyard or
                outfitting != station_lookup.outfitting or
                rearm != station_lookup.rearm or
                refuel != station_lookup.refuel or
                repair != station_lookup.repair
            ):
                if tdb.updateLocalStation(
                    station=station_lookup,
                    lsFromStar=lsFromStar,
                    blackMarket=blackMarket,
                    maxPadSize=maxPadSize,
                    market=market,
                    shipyard=shipyard,
                    outfitting=outfitting,
                    rearm=rearm,
                    refuel=refuel,
                    repair=repair
                ):
                    lines, csvPath = csvexport.exportTableToFile(
                        tdb,
                        tdenv,
                        "Station",
                    )
                    tdenv.NOTE("{} updated.", csvPath)

        # If a shipyard exists, update the ship vendor list.
        if 'ships' in api.profile['lastStarport']:
            ships = list(
                api.profile['lastStarport']['ships']['shipyard_list'].keys()
            )
            for ship in api.profile['lastStarport']['ships']['unavailable_list']:
                ships.append(ship['name'])
            db = tdb.getDB()
            for ship in ships:
                ship_lookup = tdb.lookupShip(ship_names[ship])
                db.execute(
                    """
                    REPLACE INTO ShipVendor
                    (ship_id, station_id)
                    VALUES
                    (?, ?)
                    """,
                    [ship_lookup.ID, station_lookup.ID]
                )
                db.commit()
            tdenv.NOTE("Updated {} ships in {} shipyard.", len(ships), place)
            lines, csvPath = csvexport.exportTableToFile(
                tdb,
                tdenv,
                "ShipVendor",
            )

        # Some sanity checking on the market.
        if 'commodities' not in api.profile['lastStarport']:
            print(
                'The API did not return a commodity market for this station.'
            )
            print(
                'If you think this is wrong, try again. The API will '
                'occasionally skip the market.'
            )
            return False

        # Create the import file.
        with open(self.filename, 'w', encoding="utf-8") as f:
            f.write("@ {}/{}\n".format(system, station))
            eddn_market = []
            for commodity in api.profile['lastStarport']['commodities']:
                if commodity['categoryname'] in cat_ignore:
                    continue
                if commodity['categoryname'] in cat_correct:
                    commodity['categoryname'] = cat_correct[commodity['categoryname']]
                if commodity['name'] in comm_correct:
                    commodity['name'] = comm_correct[commodity['name']]

                # Populate EDDN
                if self.getOption("eddn"):
                    eddn_market.append(
                        {
                            "name": commodity['name'],
                            "buyPrice": int(commodity['buyPrice']),
                            "supply": int(commodity['stock']),
                            "supplyLevel": EDDN._levels[int(commodity['stockBracket'])],
                            "sellPrice": int(commodity['sellPrice']),
                            "demand": int(commodity['demand']),
                            "demandLevel": EDDN._levels[int(commodity['demandBracket'])]
                        }
                    )

                f.write("\t+ {}\n".format(commodity['categoryname']))

                try:
                    commodity['stock'] = int(commodity['stock'])
                except ValueError:
                    commodity['stock'] = int(0)

                try:
                    commodity['demand'] = int(commodity['demand'])
                except ValueError:
                    commodity['demand'] = int(0)

                try:
                    commodity['demandBracket'] = int(commodity['demandBracket'])
                except ValueError:
                    commodity['demandBracket'] = int(0)


                # If stock is zero, list it as unavailable.
                if commodity['stock'] == 0:
                    commodity['stock'] = '-'
                else:
                    demand = bracket_levels[int(commodity['stockBracket'])]
                    commodity['stock'] = str(int(commodity['stock']))+demand

                # If demand is zero, zero out the sell price.
                if (commodity['demand'] == 0 or
                    commodity['demandBracket'] == 0
                   ):
                    commodity['demand'] = '?'
                    commodity['sellPrice'] = 0
                else:
                    demand = bracket_levels[int(commodity['demandBracket'])]
                    commodity['demand'] = str(int(commodity['demand']))+demand

                f.write(
                    "\t\t{} {} {} {} {}\n".format(
                        commodity['name'],
                        commodity['sellPrice'],
                        commodity['buyPrice'],
                        commodity['demand'],
                        commodity['stock'],
                    ))

        tdenv.ignoreUnknown = True

        cache.importDataFromFile(
            tdb,
            tdenv,
            pathlib.Path(self.filename),
        )

        # Import EDDN
        if self.getOption("eddn"):
            print('Posting prices to EDDN...')
            con = EDDN(
                api.profile['commander']['name'],
                'EDAPI Trade Dangerous Plugin',
                __version__
            )
            con._debug = False
            con.publishCommodities(
                system,
                station,
                eddn_market
            )

        # We did all the work
        return False

OggVorbis · Jul 28, 2015

CmdrJacks said:
You can use git to do it or there are tools out there for diff and patch...

Thanks, that's a big help!

EDIT: I'm actually getting an error here and it's not working. Log is below.

Code:

C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421>trade.py imp -P edapi -O
eddn
Traceback (most recent call last):
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\trade.py", line 104, in <module>
    main(sys.argv)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\trade.py", line 77, in main
    results = cmdenv.run(tdb)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\commands\import_cmd.py", line 114, in run
    pluginClass = plugins.load(cmdenv.plug, "ImportPlugin")
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\plugins\__init__.py", line 235, in load
    importedModule = importlib.import_module(moduleName)
  File "C:\Python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1572, in get_code
  File "<frozen importlib._bootstrap>", line 1532, in source_to_code
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\plugins\edapi_plug.py", line 92
    _agent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X)  AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12B411'  # NOQA
    ^
IndentationError: unexpected indent

OggVorbis · Jul 28, 2015

CmdrJacks said:
You can use git to do it or there are tools out there for diff and patch...

Thanks for the help, but I'm getting an error here and it's not working. Log is below.

Code:

C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421>trade.py imp -P edapi -O
eddn
Traceback (most recent call last):
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\trade.py", line 104, in <module>
    main(sys.argv)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\trade.py", line 77, in main
    results = cmdenv.run(tdb)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\commands\import_cmd.py", line 114, in run
    pluginClass = plugins.load(cmdenv.plug, "ImportPlugin")
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\plugins\__init__.py", line 235, in load
    importedModule = importlib.import_module(moduleName)
  File "C:\Python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1572, in get_code
  File "<frozen importlib._bootstrap>", line 1532, in source_to_code
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "C:\Users\JP\Desktop\kfsone-tradedangerous-f6e4298d8421\plugins\edapi_plug.py", line 92    _agent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X)  AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12B411'  # NOQA
    ^
IndentationError: unexpected indent

CmdrJacks · Jul 29, 2015

The indentation got messed up. White space is important in python. It might be the formatting the forum applies to the post or it might be the text editor you used. I'm going to go on a limb and guess you used Notepad. If this is true, try using Notepad++ (free download). It behaves better with source files. If this is still giving you trouble I'll find another way of getting it t you.

kfsone · Jul 31, 2015

I've commited 7.3.2 with a fix for the EDAPI snafu, merged some fixes from Maddavo and merged the latest data from Maddavo.

I'm still mostly offline (hospitalized) at the moment, so updates are slow.

CmdrJacks · Jul 31, 2015

Looks good, will test, hope you feel better.

LOSBOT · Aug 1, 2015

Question:
I'm looking to get started with Trade Dangerous and possibly TDHelper.
Does this all have to be installed on my gaming rig or can I just do it all from my other machines? Does it need to see ED installed for it to work?
My apologies if this was already documented.

gazelle · Aug 1, 2015

LOSBOT said:
Does this all have to be installed on my gaming rig or can I just do it all from my other machines? Does it need to see ED installed for it to work?

No, TD does not need ED. You can run it on an different computer.

<pedantic>
1st question, part 1: No
1st question, part 2: Yes
2nd question: No
</pedantic>

DRY411S · Aug 2, 2015

maddavo said:
As mentioned, the most important thing was to have all the systems with stations. The next most important thing is to have all the systems in-between systems with stations so trade routes can be considered for a run. As EDSC is capturing all star mappings (which is great - but unnecessary for TD), it's probably worthwhile editing edscupdate so that we just grab the systems within the inhabited sphere. I guess I should flag that on bitbucket.

I am mapping only the 'in-between systems' at the moment, and there are perhaps more of these than you might think. I recently made a system 'home' and asked TD to list me all the systems within 15 ly. There were 21. Using the left hand navigation panel in game, there were in fact 38.

So far, I've added 12 of them to EDSC via the web site.

Pizz001 · Aug 13, 2015

HI
i've just reinstalled TD after upgrading to windows 10 ( and i wished i'd remembered to keep a copy of the file before i did a clean install)

i've used sourcetree to pull a copy down and all used the zip as a test and i keep getting this same error when i try to run TD

C:\trade>trade.py
Traceback (most recent call last):
File "C:\trade\trade.py", line 38, in <module>
from commands import *
File "C:\trade\commands\__init__.py", line 2, in <module>
from commands.commandenv import CommandEnv
File "C:\trade\commands\commandenv.py", line 4, in <module>
from tradedb import AmbiguityError, System, Station
File "C:\trade\tradedb.py", line 744
yield from self.systemByID.values()
^
SyntaxError: invalid syntax

C:\trade>

any idea's where i've screwed up ? or as to whats giving me the error

thanks