Release TCE-Relay, a helper tool for TCE Mk. II

flat

F
Hello commanders,

I'm a big fan of Eventure's work. Honestly, I can't imaging playing Elite without TCE anymore.
For TCE Mk. II, see: https://forums.frontier.co.uk/showthread.php/223056-RELEASE-Trade-Computer-Extension-Mk-II

One thing I missed in TCE was the ability to update commodity prices by crowdsourced data (EDDN/EDDB).
More than often, I found myself flying to the same stations over and over, and all other stations had ancient commodity prices within TCE.

Therefore I've created a little helper tool for TCE:

It's called TCE-Relay and will fetch updated commodity prices for all stations in your TCE database.
Of course, it will only update prices that are newer than your local prices.

At the moment, it's still in beta (version 0.2), but at least it works for me. ;)

You can download the source code and the binaries here:
https://github.com/flattermann/TCE-Relay/releases

To be sure, please make a backup of your TCE/DB folder first!

Technically, it's a client/server solution.

On my server, I'm collecting live price updates from EDDN and store them in a database.
The client will send my server a list of known stations from the TCE DB with their last update date.
The server will send back updated prices for these stations.
The client will update the prices in the TCE DB.

Eventure will include a button for TCE-Relay in the next TCE release, but you can already try TCE-Relay:
Just unpack the ZIP and run TCE-RelayClient.exe, preferably from command line to see some output.

PS: This is my first Python project ever. If you think you can improve/optimize my code (and you probably can if you've used Python before), feel free to issue a Pull Request on GibHub:
https://github.com/flattermann/TCE-Relay

PPS: I've already posted this in the TCE thread, but I think it's a good idea to have a separate thread for it.

Since 0.3.3-beta, it includes experimental support to add new markets to TCE.
See https://forums.frontier.co.uk/showt...or-TCE-Mk-II?p=4423114&viewfull=1#post4423114


So long,
flat
 
Last edited:

Cmdr Bupu

Cmdr Bupu
relay not working

Hi Flat,

like you i'm a big fan of tce mk2 and eventure..
and I thought i'd give your program a try.

but I seem to be having problems with it.

1. installed it into D:tcerelay ( I only have windows C: on my ssd drive everything else is in d:)

pointed relay at where latest tce is only to get.

D:\tcerelay>tce-relayclient --tce-path D:\tcemkII1305
STATUS:preparing request
ERROR:Unable to create request!
Traceback (most recent call last):
File "TCE-RelayClient.py", line 353, in <module>
File "TCE-RelayClient.py", line 208, in getJsonRequest
ValueError: invalid literal for int() with base 10: '32 PM'


During handling of the above exception, another exception occurred:


Traceback (most recent call last):
File "C:\Python34\lib\site-packages\cx_Freeze\initscripts\Console.py", line 27, in <module>
File "TCE-RelayClient.py", line 356, in <module>
NameError: name 'exit' is not defined


D:\tcerelay>tce-relayclient --tce-path D:\tcemkII1305

oh and python is in D: as well but I don't think this is a issue as other python programs work ok

any ideas what I've done wrong.
cheers Richard
 

flat

F
Hi Bupu,

thanks for the feedback!

Cmdr Bupu said:
Traceback (most recent call last):
File "TCE-RelayClient.py", line 353, in <module>
File "TCE-RelayClient.py", line 208, in getJsonRequest
ValueError: invalid literal for int() with base 10: '32 PM'
Click to expand...

Oh great, it seems that TCE stores the time as string in a localized format.
TCE-Relay is expecting "HH:MM:SS" as time format, and your DB probably contains "hh:MM:SS AM/PM".

Wait, I'll prepare a new build...
 

Cmdr Bupu

Cmdr Bupu
relay not working

flat said:
Hi Bupu,

thanks for the feedback!

wow even quicker reply then eventure that is impressive..
look forward to new build.




Oh great, it seems that TCE stores the time as string in a localized format.
TCE-Relay is expecting "HH:MM:SS" as time format, and your DB probably contains "hh:MM:SS AM/PM".

Wait, I'll prepare a new build...
Click to expand...
........
 

flat

F
Ok, I hope I've fixed the problem.

Please try the updated version from:
https://github.com/flattermann/TCE-Relay/issues/1

Please run it with:
tce-relayclient --tce-path D:\tcemkII1305 --verbose > logfile.txt
and attach the logfile.txt to the issue or post the results here.

I need at least one info block trying to parse the TCE date to unix time:

Parsing TCE date: 512308 15:28:33
Parsed: 2016-08-25 15:28:33
Unix time 1472138913

and one block the other way around:

Updating LastDate for localMarketId 122 to 1472252000
Parsing unixtime to TCE: 1472252000
tceBase 512309 days, 22:53:20
tceDate 512309
tceTime 22:53:20


Thanks in advance!
Christian
 

eventure

eventure
flat said:
To all:

This should be fixed in 0.3.1-beta:
https://github.com/flattermann/TCE-Relay/releases

PS: Remember to update TCE/DB/Resources.db with the file from this link, otherwise TCE's Route Finder will crash.
Click to expand...

I have to stop this here, because the Resources.db is not compatible with the V1.3.9.1 version of TCE! Don't replace it with your current version.
I have send a mail to the author of TCE Relay to clarify this.

EDIT: PROBLEM SOLVED!
 
Last edited:

flat

F
Eventure said:
I have to stop this here, because the Resources.db is not compatible with the V1.3.9.1 version of TCE! Don't replace it with your current version.
I have send a mail to the author of TCE Relay to clarify this.
Click to expand...
Sorry for the misunderstanding, I wasn't aware of that.

I'll upload a new TCE-Relay version that will not require an updated Resources.db.
 

flat

F
Cagman said:
I am using TCE Mk.II with E:D Market Connector.

Do I need this tool in order to have updated prices on Commodities?
Click to expand...

EDMC updates the prices for your current station (where you are docked at the moment) using Elite's Companion API.

TCE-Relay updates the prices of all of your stations (using crowdsourced data, generated by e.g. EDMC users)
 

flat

F
I just released TCE-Relay 0.3.3-beta:
https://github.com/flattermann/TCE-Relay/releases

It includes experimental support to add new markets to TCE.


In the TCE 1.3.9.1 this will not work 100%, because TCE will not recognize the new markets automatically on docking.
You should definitely make a backup of your TCE/DB folder before trying this feature.
And you should not use it on your production database because it will probably prevent DB upgrades for the next TCE release.
If you add too many markets to TCE, things will slow down significantly (using 500-1000 markets works on my system).


If I haven't scared you away, you can add a single market like this: :)


TCE-RelayClient.exe --i-know-the-risks -a "STATIONNAME@SYSTEMNAME"
e.g.
TCE-RelayClient.exe --i-know-the-risks -a "ABRAHAM LINCOLN@SOL"


or you can add all markets around a star:


TCE-RelayClient.exe --i-know-the-risks -A "SYSTEMNAME,LY,LS,PLANETARY"


SYSTEMNAME=System Name
LY=Max Range from given System (in LY)
LS=Max Distance from star (in LS)
PLANETARY=Y/N, Y will also add Planetary Bases


e.g.
TCE-RelayClient.exe --i-know-the-risks -A "SOL,25,500,N"
will add all markets within 25LY of SOL, where the market is max. 500LS away from its sun, without Planetary Bases.


This will also fetch the current prices for these markets and hence make them available to TCE Route Finder.




When you first visit a station that you've added this way, TCE will show it as unknown (ie. SECTOR xy).
You need to:
- Click on "Add market" in TCE
- Click on "Modify Entry"
- Set "System Sector" to correct value as shown in TCE
- Click "Actual Coordinates"
- Click Save


Good luck ;)
/flat
 
Last edited:

Cmdr Bupu

Cmdr Bupu
relay updates

Not to be against this great bit of software...

but does this version not go against eventure's idea that only stations you have visited should be included in your own database.

updating your own stations I feel is ok but adding places you have not been to is just using the crowdsourced data.

but saying this I also feel that in the future you would be able to communicate with stations in the same system and request prices without actually visiting them..eg like internet shopping.
so i'm not sure what i'm saying now... (unless it will only deal with markets in your current system)

Discuss.....
ps. I know you can always just not use it.. buts its there to use .....



flat said:
I just released TCE-Relay 0.3.3-beta:
https://github.com/flattermann/TCE-Relay/releases

It includes experimental support to add new markets to TCE.


In the TCE 1.3.9.1 this will not work 100%, because TCE will not recognize the new markets automatically on docking.
You should definitely make a backup of your TCE/DB folder before trying this feature.
And you should not use it on your production database because it will probably prevent DB upgrades for the next TCE release.
If you add too many markets to TCE, things will slow down significantly (using 500-1000 markets works on my system).


If I haven't scared you away, you can add a single market like this: :)


TCE-RelayClient.exe --i-know-the-risks -a "STATIONNAME@SYSTEMNAME"
e.g.
TCE-RelayClient.exe --i-know-the-risks -a "ABRAHAM LINCOLN@SOL"


or you can add all markets around a star:


TCE-RelayClient.exe --i-know-the-risks -A "SYSTEMNAME,LY,LS,PLANETARY"


SYSTEMNAME=System Name
LY=Max Range from given System (in LY)
LS=Max Distance from star (in LS)
PLANETARY=Y/N, Y will also add Planetary Bases


e.g.
TCE-RelayClient.exe -A "SOL,25,500,N"
will add all markets within 25LY of SOL, where the market is max. 500LS away from its sun, without Planetary Bases.


This will also fetch the current prices for these markets and hence make them available to TCE Route Finder.




When you first visit a station that you've added this way, TCE will show it as unknown (ie. SECTOR xy).
You need to:
- Click on "Add market" in TCE
- Click on "Modify Entry"
- Set "System Sector" to correct value as shown in TCE
- Click "Actual Coordinates"
- Click Save


Good luck ;)
/flat
Click to expand...
 

flat

F
Well, I can't speak for Eventure, but I received the impression that he is fine with this feature.

Also, it's not meant to be used as an "add-all-stations-to-TCE" feature.

The idea is that you can easily add some stations e.g. around your current position.

Ok, right now it's just an experimental proof-of-concept and I'm not checking that; theoretically you could add the entire universe to your DB at the moment.

But you'll realize very soon that practically this was a bad idea... ;)
 

eventure

eventure
I have no problem with such a mod for TCE. Updating the prices for visited stations was badly needed and the next logical step in the development of TCE.
But I am laking of skill to realize it myself, so I am quite happy with flat, that he took the torch and realizing it on his deck, setting up and maintaining the server, etc.

I can't prohibit his wish, to add market data to TCE, he didn't gained himself, nor I would, as long as it doesn't disturb TCE's core functions, gathering data. I like to discover things in game,
so I would not use his "Add Markets" option for myself. But some or more requesting a similar feature of TCE since day 1, so I am really curious in how many users are interested in such a technic.
 
Last edited:

cpw

C
Eventure, I used it to add everything within about 50ly of a random spot I happened to be, just so I could go trade hunting. It worked like a charm. I do try and hit up markets whenever I can, but building a useful corpus of data seems to be the work of many months and I'm an intermittent player at best.
 

hitchhiker54

hitchhiker54
I haven't done trading in a while, but I used to scrape star data for 600ly radius of Sol from EDSM, and add the price data from maddavo's site for TradeDangerous to chew on. Great for scope but my 6700K needed a good amount of time to find routes. A more limited approach can be nice :)
 

Cmdr Bupu

Cmdr Bupu
am i doing this right

ok thought id try to use this to add station markets in my current system.

am I doing this right
I am in orcus at the moment docked at Tombaugh Station

my tce is in d:\tce and relay is in d:\tcerelay

so
D:\tcerelay>tce-relayclient --tce-path D:\tce -A "ORCUS,1,2000,N"

should give me 1 ly around orcus (so not including other systems) and 2000ls from orcus star without planet bases

but I get

D:\tcerelay>tce-relayclient --tce-path D:\tce -A "ORCUS,1,2000,N"
STATUS:preparing request
Requesting data for 0 markets
getJsonRequest took 0.000628118672047813 seconds
STATUS:Sending request
Compressed JsonRequest from 153 to 135 bytes
200
{'Server': 'nginx/1.6.2', 'Connection': 'keep-alive', 'Date': 'Sat, 03 Sep 2016 02:15:54 GMT', 'Transfer-Encoding': 'chunked', 'Content-Encoding': 'gzip', 'Content-Type': 'application/json'}
sendRequest took 0.70618280387031 seconds
STATUS:processing response
ServerProcessTime 0.0007260000000002265
Got prices for 0 markets
processJsonResponse took 0.0018302834454925199 seconds
Updated 0 stations with 0 prices
Total runtime: 0.7134750015198776 seconds


D:\tcerelay>
i am using the correct version
D:\tcerelay>tce-relayclient --version
0.3.3-beta

and tce version 1391

D:\tcerelay>
 
You must log in or register to reply here.
Top Bottom