Confirming, Confirmed? - how the bug reporting system hides common bugs

[Ian Doncaster]

The bug reporting system starts off with bugs in Confirming state, and moves them to Confirmed after a certain number of corroborations are received (around 10 "can reproduce" appears to do it). The threshold is quite high in practice - there are only 30 Confirmed bugs (many pushed there by the organised AX groups), and just over 1,000 Confirming. However, there are several bugs which have been corroborated with well over 10 reports and are not yet Confirmed ... because those corroborations are spread out over multiple individual reports.

For example, the "FSS display distorts diagonally" bug has been reported at least 10 separate times - normally enough for Confirmed if they'd all been on one bug - but even though many of those individual reports also have corroboration on them, none of them has yet broken through the threshold. https://issues.frontierstore.net/issue-detail/960 has a list of the other reports.

Similarly the "planets still show incorrectly as mapped, despite the claim it was fixed" bug has at least 11 separate reports, one of which has seven confirmations on it ... still, none of them are Confirmed. https://issues.frontierstore.net/issue-detail/284 has a list of the other reports.

There are plenty of other bugs with similar multiple reporting - the "CZs/POIs run away" bug, the "dispersed third rings are invisible" bug, the "skimmer combat crash" bug (15 confirming, 0 confirmed), the "war zones hang around days after the war is over" bug etc. - with little hope of getting to Confirmed.

The problem is:
1) The search for similar bugs at step 2 of reporting rarely works - usually it returns nothing at all for me, occasionally it returns some completely unrelated stuff.
2) This means that duplicate bugs are very likely to be filed.
3) This means that confirmations from people who do a bit more of a search are going to be spread out across multiple bugs (no-one has time to stick a confirmation on every duplicate they find, or even look for duplicates past the first - I've only done it for a couple of the bugs above to make the pattern clear)
4) Which means that the longer it goes on, the worse it gets - confirmations get spread out across an increasing field of bugs, and none of them get enough to break through

If Frontier are expecting Confirmed to be the only important bugs, then they need to either stop doing that or bring the threshold for Confirmed way down - seriously, if three people say it's a bug (one reports, two corroborate) then it's either a bug, or it's a common misunderstanding that should be marked as "not a bug" with some sort of explanation or at least a "working as intended" comment.

Conversely, if Frontier are still going through the Confirming bugs as they used to go through the bug report forums, that's great - but they need to be manually marking them as Confirmed+Acknowledged (or Duplicate) when they do, so that we can see they're doing that (and can therefore see the bug more easily as Confirmed and not waste everyone's time with another duplicate report)

 

[Oopsie Daisy]

Meh, I stopped reporting bugs. Just too many of them. And I am not willing to be unpaid Frontier QA. The hell with Agile!

 

[Factabulous]

Meh, I stopped reporting bugs. Just too many of them. And I am not willing to be unpaid Frontier QA. The hell with Agile!

Lol - I do love this non sequitur : fdev release every few months they are certainly not practicing agile.

Back on-topic: they need to improve the search if people aren't finding the bugs (though seriously - how on earth 10+ people failed to find the restock bugs I have no clue), and QA need to start merging the duplicates.

 

[CMDR Cosmic Spacehead]

Yeah, the search function is useless.

I ended up scrolling first through confirmed bugs to vote up, and add to, the major ones I was experiencing.
Then scrolling down confirming bugs to add my own details too.
Then I don't look at the rest. Lol

 

[RexKraemer]

This I wrote elsewhere:

But actually, this issue tracking and upvoting thing FDev has put there, ... I don't have words for it. I am a software developer, if I would say to my users, hey, ok this bug is obvious but I don't care until you bring me some more people to support it, I do nothing... shakes head in disbelieve

FDev, this is how you should do it:

• who ever finds a bug it can be documented using your issues web site
• you do some inspection, check if not already reported, merge if it is, ... keep it clean stuff (here we could assist like the moderators here in the forum do)
• you give every issue two tags:

• priority, like: doesn't matter, ..., important, ..., game breaking
• difficulty, like: easy, ..., normal, ..., hard, risky and many side effects

(By the way, this tagging would already be important and good professional communication to us.)

Now you can easily do bug fixing: easy and game breaking first, hard and doesn't matter last if ever.

You could set up continuous update delivery with that, as you like it: daily, weekly or what ever with the easy bug fixes, monthly are half a year with more testing for the hard and risky ones. Again we could assist with testing, like, if I want, I subscribe into the beta/alpha update stream, and I get all the untested bug fixes directly downloaded incl. some update informations like links to the issues, to report back.

This issue tracking grind game they installed is a shame for a software development company.

 

[RexKraemer]

The hell with Agile!

There is nothing agile in this!
Is there something I missed? Did FDev talked about "agile" when installing this? Can someone elaborate, link?

 

[Jaggid Edje]

The search function has worked for me for the handful of bugs I've reported. I acknowledge it could be better, but I also posit some of it is people not being intelligent about what they search for and, in many cases, not searching at all. Forum threads started on a topic where another thread with almost the same subject line is on the very first page is a very common thing in all games I don't doubt that lack of any search is pretty common.

Taking the restock bug for example. If you search the tracker for "restock" you get pretty much every one of those reports. Now, that would be the intelligent keyword to search for with that issue. Clearly, the folks that started each of those (except the first guy, of course) either did not search at all, or did not search intelligently.

 

[Riverside]

I have yet to raise a new report, I'm just confirming ones I can confirm.

Duplicates used to get merged on the old system, presumably the same staff will perform that role for these too.

 

[Ian Doncaster]

The search function has worked for me for the handful of bugs I've reported.

Sorry, I might not have been clear enough. The search filter on the "show me a list of bugs" page like https://issues.frontierstore.net/reported-issues/elite-dangerous works fine - that's how I counted the "skimmer" bugs, for example. And that's presumably where each one of these duplicates is picking up its 0-3 corroborations, from people using that search and finding one of the copies.

It's the "Check" stage of "Submit Report" where it shows - or more usually doesn't - similar reports, where it really needs to for this to work.

 

[RexKraemer]

The search function has worked for me for the handful of bugs I've reported. I acknowledge it could be better, but I also posit some of it is people not being intelligent about what they search for and, in many cases, not searching at all. Forum threads started on a topic where another thread with almost the same subject line is on the very first page is a very common thing in all games I don't doubt that lack of any search is pretty common.

Taking the restock bug for example. If you search the tracker for "restock" you get pretty much every one of those reports. Now, that would be the intelligent keyword to search for with that issue. Clearly, the folks that started each of those (except the first guy, of course) either did not search at all, or did not search intelligently.

It should not be our responsibilty to do it right. For us users/customers/players it is good enough to report bugs! FDev should do everything to make reporting bugs as easy as possible and with any reported bug FDevs responsibility starts.

 

[Jaggid Edje]

Sorry, I might not have been clear enough. The search filter on the "show me a list of bugs" page like https://issues.frontierstore.net/reported-issues/elite-dangerous works fine - that's how I counted the "skimmer" bugs, for example. And that's presumably where each one of these duplicates is picking up its 0-3 corroborations, from people using that search and finding one of the copies.

It's the "Check" stage of "Submit Report" where it shows - or more usually doesn't - similar reports, where it really needs to for this to work.

Ahh..yah, that check stage is horrible. it makes you type a longer search than is useful for the kind of parsing that the search function appears to do. They either need to redo the way it parses what you type or let you use less words so that it can get you good results. I totally agree with you there.

 

[Jaggid Edje]

It should not be our responsibilty to do it right. For us users/customers/players it is good enough to report bugs! FDev should do everything to make reporting bugs as easy as possible and with any reported bug FDevs responsibility starts.

Of course it is partially our responsibility to "do it right". When you report ANYTHING in real life, you have to report it intelligently or it is of no value and you are wasting the time of both yourself and the party you are reporting it to.

Ask any customer service or 911 operator what I'm talking about if you still do not understand.

You are quite correct that FDev is responsible for making sure that reporting the issues as easy as possible though.

 

[RexKraemer]

Of course it is partially our responsibility to "do it right". When you report ANYTHING in real life, you have to report it intelligently or it is of no value and you are wasting the time of both yourself and the party you are reporting it to.

Ask any customer service or 911 operator what I'm talking about if you still do not understand.

We should only do it right in a minimalistic way. Only what is necessarily needed, no tedious searching, commenting, whatever. I thought this is clear.

As it is now FDev is wasting our time when they expect that we should avoid doubled reports, comment it so that it gets some priority, and still, they can just let it stand there forever.

 

[Oopsie Daisy]

There is nothing agile in this!

Maybe that is FDev's problem.

"Don’t be afraid to make changes" - FDev is not, obviously LOL.
"Working software is the primary measure of progress" - clearly FDev fails here
"Attain customer satisfaction through continuous delivery of software" - fail again

 

[Old Duck]

Meh, I stopped reporting bugs. Just too many of them. And I am not willing to be unpaid Frontier QA. The hell with Agile!

Me too, even though I was willing to be Frontier's unpaid QA department. But when they ignore my detailed reports for over a year (or worse, pay lip service but never actually fix any of the bugs), and then they foist this monstrosity of a bug reporting system on us, well I just say SCREW 'EM

 

[Oopsie Daisy]

Me too, even though I was willing to be Frontier's unpaid QA department. But when they ignore my detailed reports for over a year (or worse, pay lip service but never actually fix any of the bugs), and then they foist this monstrosity of a bug reporting system on us, well I just say SCREW 'EM

Yup. Ignored, detailed bug reports are something I also got fed up with. Fdev can do their own QA work, or not, as far as I'm concerned... non sequitur or otherwise lol.

 

[Dutchman141]

Considering the high number of new, old and very obvious bugs that came with the last update I feel like Fdev was practically counting on our QA contribution.
I reported a few bugs but thank you very much I'll take a pass on being their tester.
They're QA leaves much to be desired in general but the last update was simply sloppy work.

The bug tracker system is nothing but a reporting system to me, I don't see any value in this voting system. Bugs need to be fixed, period.

 

[Majic-12]

I can't speak for anyone else, but if a bug reporting system requires me to be intelligent to succeed, then it has already failed.

 

[Deleted member 38366]

That issue tracker?

They lost me when I realized I needed a highly unsafe Javascript-enabled Browser just to open it.

Other than that, my offer still stands. 50$ per documented and reproduced/confirmed bug (1st Discovery/1st to report) should do.
I'd say that's a fair rate for "Freelance/Independent QA specialist".

They don't want to pay up for outsourcing QA? Their problem

 

[Drelthar]

Nothing loads on the issue tracker for me.

I just get "This issue could not be loaded" every time.

Anyone know what that's about?

I mean, I can view their titles, but then clicking them shows that message. Also, you can only vote on four issues at any one time?!?