Unexpected influence results reporting

Schlack

S
FD have requested that we report specific examples of BGS weirdness, with specific information, into a specific thread to help them diagnose problems.

link to instructions and thread here:

https://forums.frontier.co.uk/showt...e-the-update?p=7365647&viewfull=1#post7365647



Letting us know that you're not seeing the expected results after Saturday's tick from your actions is useful feedback, but we need specific information to enable us to look into your particular circumstances.

To look at an unexpected faction result, such as "We pushed a lot of +inf% to Faction A through missions, bounties & trading, Faction B and C had a bit of +inf% in the same system from bounties, so Faction A didn't increase as much as we'd expect", as a minimum we need to know:
- the faction
- the starsystem
- the approximate date when you saw the unexpected result

In addition, to consider your feedback when we're investigating the balance of particular actions, such as a mission result affecting an unexpected faction, or some exploration data sales not having a positive effect then we'd also need:
- all of the above AND
- your commander name or names
- the approximate date / time or location of your actions
 

MottiKhan

MottiKhan
Dommarraa said:
No.
I have wasted enough of my life on 3.3, and reported plenty. I can understand if they don't know how to fix it, but if they can't even replicate these issues they are not fit to be developing any of this game, let alone rewriting the bgs.
Click to expand...

+1 rep. I totally agree. FD should try playing their own game and work it out.

That, or pay some alpha testers.
 

Factabulous

Factabulous
Easy enough with limited skills to cull real data for FDev from your journals. or just noting what you did when, and to whom.

I know from working in s/w that saying 'X happened at Y time after doing Z' gets much better responses than, 'well my faction feels that things are weird', which is about all the information that they are getting on a lot of BGS tickets.

I always put the vague, underspecified bugs to the back of the queue in the hope that someone would actually document what was happening so I could look into it.
 

Deleted member 38366

D
I've essentially abandoned the BGS for example.

The one thing I've suggested a very long time ago (basically what every single professional MMO company would do) :
- before implementing larger BGS changes, they need to store 2-4 weeks of BGS Data as a backup
- they code their new or changed BGS
- then they shall feed the "cold" Datasets into the new BGS and compare results with the known ones from the old BGS side-by-side
- fully automated tests scan the new Results for Anomalies and specifically test if the observed changes match the intended behavior

Instead, they make huge changes to a very delicate piece which ten thousands of Players sink insane amounts of work hours into. Without any Alpha Testing (looks more like 'If it compiles without errors - we'll let it go live").
Then they encounter massive fleet problems but instead of running the required automated tests, they fully rely on isolated, singular Bug Reports.
Doesn't work. It might, but it takes forever and then some before even the biggest issues are identified or fixed.

Delivering a completely broken blackbox BGS, then applying blackbox changes and asking BGS Players to test? Test what?
Testing requires two things :
- how - precisely - is it supposed to work
- then with that knowledge make calculated Inputs and confirm vs. observed BGS Outputs, within constraints of available Benchmark Systems (other Traffic/other BGS Inputs)

IMHO :
If they still haven't understood what the main issues are and what they did by now, then sorry... they'll never get it.

The whole thing was FDev's job right from the start, they botched it good - but now they're asking Players to invest alot of work for "Sample data" FDev has easy and live access to at all times throughout the Galaxy?
Doesn't make sense. They know the BGS Inputs of every single Player and they have all the results on their own Servers. It's their job, not the Player's job. No more "free Alpha Testing", done with that crap.
And without "Commander's Intent" - how are things supposed to work, no point in Testing or Reporting anyway, without having clearly defined what is Working as Intended and what might be a bug.
 
Last edited by a moderator:

Ian Doncaster

Ian Doncaster
FalconFly said:
Doesn't make sense. They know the BGS Inputs of every single Player and they have all the results on their own Servers.
Click to expand...
There's some evidence that this may be the specific problem and why they haven't fixed it yet.

Check transactions for X system: 10 transactions logged against faction A
Check results for X system: faction A influence decrease
Conclusion: everything is working fine
Reality: those 10 transactions were standard trade runs and should have been positive for faction A

If the problem is that the influence contribution is getting inverted too early in the process, though, you'd never spot a problem from that sort of check, unless the player making those 10 transactions said what they were, and then they didn't match up at all with what was recorded on the server about them. Presumably that's why they've asked for specific information so that they can determine where the misrecording is happening.
 

Factabulous

Factabulous
FalconFly said:
The whole thing was FDev's job right from the start, they botched it good - but now they're asking Players to invest alot of work for "Sample data" FDev has easy and live access to at all times throughout the Galaxy?
Doesn't make sense. They know the BGS Inputs of every single Player and they have all the results on their own Servers. It's their job, not the Player's job. No more "free Alpha Testing", done with that crap.
Click to expand...

They have all the data, so much that finding any one thing is not easy. It's like pointing at a haystack and telling them to find a needle. Pointing at a time/person/place lets them filter down and see what happened there, and if it makes sense.

Otherwise they look at the whole BGS and say, well, there are wars, there are outbreaks, things are moving around, what's wrong? The BGS possibly looks ok at the macro level, but at the micro level it's broken.

I agree with other comments that the BGS is eminently testable, it is pretty concerning if FDev don't have better tests on how it functions.

But if we want to move forward and get it fixed (which is a benefit to us, not to Fdev) then pointing them at specifics is the way to so that.
 

Schlack

S
Factabulous said:
They have all the data, so much that finding any one thing is not easy. It's like pointing at a haystack and telling them to find a needle. Pointing at a time/person/place lets them filter down and see what happened there, and if it makes sense.

Otherwise they look at the whole BGS and say, well, there are wars, there are outbreaks, things are moving around, what's wrong? The BGS possibly looks ok at the macro level, but at the micro level it's broken.

I agree with other comments that the BGS is eminently testable, it is pretty concerning if FDev don't have better tests on how it functions.

But if we want to move forward and get it fixed (which is a benefit to us, not to Fdev) then pointing them at specifics is the way to so that.
Click to expand...


This^^^

FD have the top down macro perspective, we have the bottom up micro perspective. It is quite possible that we are seeing different pictures!
 

Deareim

Deareim
Dommarraa said:
No.
I have wasted enough of my life on 3.3, and reported plenty. I can understand if they don't know how to fix it, but if they can't even replicate these issues they are not fit to be developing any of this game, let alone rewriting the bgs.
Click to expand...

+1.
Wasted time on bug report during beta and after update with details for NADA. But compared to Dommaraa, I don't understand how they cannot how it is working. For the rest I agree with him.

And even if I wanted to do it, I don't know what are the expectations.
 

Deareim

Deareim
Factabulous said:
They have all the data, so much that finding any one thing is not easy. It's like pointing at a haystack and telling them to find a needle. Pointing at a time/person/place lets them filter down and see what happened there, and if it makes sense.

Otherwise they look at the whole BGS and say, well, there are wars, there are outbreaks, things are moving around, what's wrong? The BGS possibly looks ok at the macro level, but at the micro level it's broken.

I agree with other comments that the BGS is eminently testable, it is pretty concerning if FDev don't have better tests on how it functions.

But if we want to move forward and get it fixed (which is a benefit to us, not to Fdev) then pointing them at specifics is the way to so that.
Click to expand...

I am sorry but I don't find excuses to FD. BGS is on the path of PP. Living with bugs.
If you have no clue how your software is build and how you can tested, then maybe you can change jobs, both from developer and tester perspective.

Schlack, who has started this thread, has said even during the beta it would be a ****show if they continue like this. And they did. But hopefully, they have 100 people working on it.
 
Last edited:

Rubbernuke

Rubbernuke
Another example of oddness, or simply poor UI- I have a faction that has two down arrows (red) and yet the INF goes up....what does that tell me? In some systems INF acts as if its linked to these sliders, while in other areas it acts independently.
 

Deleted member 38366

D
Ian Doncaster said:
There's some evidence that this may be the specific problem and why they haven't fixed it yet.

Check transactions for X system: 10 transactions logged against faction A
Check results for X system: faction A influence decrease
Conclusion: everything is working fine
Reality: those 10 transactions were standard trade runs and should have been positive for faction A

If the problem is that the influence contribution is getting inverted too early in the process, though, you'd never spot a problem from that sort of check, unless the player making those 10 transactions said what they were, and then they didn't match up at all with what was recorded on the server about them. Presumably that's why they've asked for specific information so that they can determine where the misrecording is happening.
Click to expand...

And exactly these fundamental issues (i.e. flipped flags) would be easily visible in automated Testing. 10 Positive Inputs sure would not reduce Influence after all.
After all, the Automatron knows the Inputs were positive as it remembers its avenue, type and intensity.
The automatically highlighted Bug Analysis then would point to the issue - where a Dev would realize positive Trade Inputs were bugged and counting as negative instead.

99.9% of all BGS bugs and issues would be discovered within 5 Minutes or less during such automated and isolated Test-runs, incl. deeper issues such as forced (unintended) Equilibrium (stale state with unintended strong positive stability in an environment that should be fully dynamic), undesired spikes (undesired combination of effects adding to each other) or insufficient output (Player Input not being properly reflected as Influence Output).

That's the joy of automated Testing. You get and can Analyze a Billion BGS Samples within a few Minutes - each having 100.0% benchmark quality - and can see if the displayed results reflect the intended results.

There's rather plenty of evidence not even rudimentary manual Alpha Testing was conducted, let alone in any automated way.
That Testing was and remains entirely left to the Players, which is the root cause and resulted in the BGS more or less blowing up.
 
Last edited by a moderator:

Schlack

S
Automated testing simply cant replicate the sometimes insane types of activities and manner in which theyre done of the playerbase!
 

Dommarraa

Dommarraa
Schlack said:
Automated testing simply cant replicate the sometimes insane types of activities and manner in which theyre done of the playerbase!
Click to expand...

No, it's bad at finding edge cases, but they are not the issue. It's basic 1 action producing wrong influence direction.
 
You must log in or register to reply here.
Back
Top Bottom