Page 201 of 201 FirstFirst ... 196199200201
Results 3,001 to 3,010 of 3010

Thread: TradeDangerous: power-user trade optimizer

  1. #3001
    Originally Posted by MarkAusten View Post (Source)
    I have a work around for the issue for the moment bu in my investigations I note that SQLite is set up not to use FK constraints by default, it is probably a good idea either to enable the FK constraints ("PRAGMA foreign_keys=ON") and ensure that the values are correctly set or leave the FK constraints off and remove the FK definitions from the database schema. The former is the preferred option, obviously, but may involve a lot of work to ensure the referential integrity.
    Just as a point I think it's important to make - we (by which I really mean eyeonus) are not in the business of making big changes to TD. The plan was and still is to have a method of getting data into the existing TD, with minimal changes/work in a post-maddavo world. At present we have one stupid bug which is hard to pin down (https://github.com/eyeonus/EDDBlink-listener/issues/7), but once that it resolved, then we will be ready to release. Mission creep into messing with the TD database in any way which "may involve a lot of work" is not one of the project deliverables.

  2. #3002
    Originally Posted by Tromador View Post (Source)
    Just as a point I think it's important to make - we (by which I really mean eyeonus) are not in the business of making big changes to TD. The plan was and still is to have a method of getting data into the existing TD, with minimal changes/work in a post-maddavo world. At present we have one stupid bug which is hard to pin down (https://github.com/eyeonus/EDDBlink-listener/issues/7), but once that it resolved, then we will be ready to release. Mission creep into messing with the TD database in any way which "may involve a lot of work" is not one of the project deliverables.
    Fair point. However I reserve the right to raise it again should it become and issue that I can't solve with a work-around in future.

    Frankly, if I knew python better I'd take a look myself but that programming language is not one that I wish to be familiar with.

  3. #3003
    Originally Posted by Tromador View Post (Source)
    Just as a point I think it's important to make - we (by which I really mean eyeonus) are not in the business of making big changes to TD. The plan was and still is to have a method of getting data into the existing TD, with minimal changes/work in a post-maddavo world. At present we have one stupid bug which is hard to pin down (https://github.com/eyeonus/EDDBlink-listener/issues/7), but once that it resolved, then we will be ready to release. Mission creep into messing with the TD database in any way which "may involve a lot of work" is not one of the project deliverables.
    Anything we quasi-programmers can do to help nail that problem down?

  4. #3004

  5. #3005
    Originally Posted by MarkAusten View Post (Source)
    Fair point. However I reserve the right to raise it again should it become and issue that I can't solve with a work-around in future.

    Frankly, if I knew python better I'd take a look myself but that programming language is not one that I wish to be familiar with.
    If we change a load of stuff, maybe - but in this case, the only change was a (now reverted) kludge to workaround a problem created for us. If (as is likely) we've changed nothing, then I imagine we would refer you to the python manual and the source code, like it or not!

  6. #3006
    Originally Posted by Avi0013 View Post (Source)
    Anything we quasi-programmers can do to help nail that problem down?
    Mebbe... if you find one where it's not working, you might try testing the SQL manually.

    If you see my comment here you could insert the code snippet to debug the SQL - lots of output, so be warned - then use that debug output to give the underlying SQL commands. Then try those manually on the database through a DB app which lets you run arbitrary SQL commands. See if they also fail. Read on in that thread about the INPUT vs UPDATE statements and what/why the difference, so you know what you need to see. Then let us know what happened.

    Other than that, point any python guru friends at it and see if they can see something untoward? If eyeonus or my testing showed up anything obvious it would already be fixed. It's rather vexing.

  7. #3007
    Originally Posted by Tromador View Post (Source)
    Mebbe... if you find one where it's not working, you might try testing the SQL manually.

    If you see my comment here you could insert the code snippet to debug the SQL - lots of output, so be warned - then use that debug output to give the underlying SQL commands. Then try those manually on the database through a DB app which lets you run arbitrary SQL commands. See if they also fail. Read on in that thread about the INPUT vs UPDATE statements and what/why the difference, so you know what you need to see. Then let us know what happened.

    Other than that, point any python guru friends at it and see if they can see something untoward? If eyeonus or my testing showed up anything obvious it would already be fixed. It's rather vexing.
    I wonder if it is the use of the exception on insert that is the cause of the problem. For testing purposes it would be interesting to see what happens if you use the much slower method of checking to see if the record exists and then inserting or updating as appropriate. If that works then it narrows the problem down to the use of the exception as decision logic and that perhaps being slightly different between different computers.

  8. #3008
    Originally Posted by MarkAusten View Post (Source)
    Has something changed with the Trade Dangerous database schema or data? With the tradedangerous.db file from Thursday last week TD Helper runs fine but if I update the database or delete it and create a new one, TD Helper throws an error:

    Code:
    Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
    The SQL command being run is:

    Code:
    select sys.name as sys_name, stn.name as stn_name 
    from System sys 
    left join Station stn on sys.system_id = stn.system_id
    I have a work around for the issue for the moment bu in my investigations I note that SQLite is set up not to use FK constraints by default, it is probably a good idea either to enable the FK constraints ("PRAGMA foreign_keys=ON") and ensure that the values are correctly set or leave the FK constraints off and remove the FK definitions from the database schema. The former is the preferred option, obviously, but may involve a lot of work to ensure the referential integrity.
    I literally have no time to work on this, but I'm fairly certain that yes it is. IIRC, there's a few pragmas that are run in the cache.py file, and turning on foreign keys is one of them.

    EDIT: Sorry looks like it's in tradedb.py:
    Code:
        def getDB(self):
            if self.conn:
                return self.conn
            self.tdenv.DEBUG1("Connecting to DB")
            conn = sqlite3.connect(self.dbFilename)
            conn.execute("PRAGMA foreign_keys=ON")
            conn.create_function('dist2', 6, TradeDB.calculateDistance2)
            return conn
    EDIT2: It's also turned on when (re)building the cache, which is done in the buildCache method in cache.py

  9. #3009
    Originally Posted by MarkAusten View Post (Source)
    I wonder if it is the use of the exception on insert that is the cause of the problem. For testing purposes it would be interesting to see what happens if you use the much slower method of checking to see if the record exists and then inserting or updating as appropriate. If that works then it narrows the problem down to the use of the exception as decision logic and that perhaps being slightly different between different computers.
    I'm not going to say it isn't possible, but I highly doubt it. The stations that are failing to update all make it to the UPDATE command. That is, the "do INSERT if IntegrityError do UPDATE" never fails to get to the "do UPDATE" part.

  10. #3010
    Originally Posted by eyeonus View Post (Source)
    I'm not going to say it isn't possible, but I highly doubt it. The stations that are failing to update all make it to the UPDATE command. That is, the "do INSERT if IntegrityError do UPDATE" never fails to get to the "do UPDATE" part.
    But you never start the transaction yourself. What if the internal transaction-algorythums of the sqlite3 module does an rollback if an IntegrityError is raised?

Page 201 of 201 FirstFirst ... 196199200201