Proposal Discussion Bugs & Issue Tracker

[rootsrat]

This is not meant to be a rant, but more of a criticism. I hope it comes across like that. Well, maybe I'll rant just a little, to vent

I think it is fair to say that despite lots of bugs being fixed, there are still way too many of them to be able to fully enjoy the game. Almost every aspect of Elite Dangerous has bugs - sometimes less, sometimes more of them - which is sort of normal in todays gaming world.

Considering the scope of Elite specifically, I don't think anyone expects a totally bug-free experience, however SO MANY bugs are very old and nothing seems to be happening about them for years! The issues logged on the tracker are not moving or being updated, to the point that I have now used up all votes on 4 different accounts and I've not been able to vote for any new issues since I don't even remember when - because absolutely nothing is happening on the Tracker with the issues I have already voted for.

Other issues just expire, because there are not enough confirmations, and this is mainly due to plethora of duplicates being logged, because the tracker app does a very bad job of finding related issues when you try to log one. With all due respect, it is just a badly designed system in my opinion.

Again, just my opinion, but this is really getting out of hand. I love your game FDEV and I love to support it and play it, I love to support you by spending money on Arx across all of my accounts. I also consider myself quite an understanding and reasonable person, but this is becoming really annoying to the point of being unacceptable from a customer point of view.

The issue tracker improvements were announced years ago... but absolutely nothing has been done about it, at least form user perspective.

I realise you guys had A LOT on your plate since Odyssey release and I know that new content is incoming, but maybe it is time to focus on fixing the game, rather than expanding it with more buggy content? (Just look at this thread, which neatly lists some (but not all!) of the fresh bugs related to Thargoid War.

And maybe its time to finally do something about that issue tracker? It is very frustrating and I personally wouldn't mind another "Beyond" kind of series of updates, where you focus more on fixing the game, than on the new content.

There, rant over! I mean critique

 

[Ozric]

It's actually worse than it has been in a long time. They haven't even bothered doing a cull of unconfirmed issues. After ever major update someone manually closes issues that are older than 30 days and haven't met the confirmation requirements, or been manually confirmed. The number of confirming issues drops from thousands to tens/low hundreds. But currently we sit at 3,366.

It hasn't been done since Update 13, you can tell because the most recently created issue that has expired is June 12th 2022.

Despite everything, I have still reported 88 issues! I can't tell you how many I've contributed to because that screen hasn't worked for years!

It is so frustrating and demoralising, I can't even convey it.

 

[[VR] Mad Mike]

I realise you guys had A LOT on your plate since Odyssey release and I know that new content is incoming, but maybe it is time to focus on fixing the game, rather than expanding it with more buggy content? (Just look at this thread, which neatly lists some (but not all!) of the fresh bugs related to Thargoid War.

to be honest... FD have used that excuse for 2 large lulls in content and really it did very little at least on the surface, to fixing bugs......... other than have 2 long periods with out much new content.

so given it seems we have a choice of new content and bugs or
no new content and bugs

I think i would rather FD carry on bringing out new content in the hope that those fixing bugs are not the same people as those creating content so may as well try to put out the fires in the content updates.

 

[Martind Forlon]

... I've started this post maybe ten times, but in the end I only want to say that personally I would really appreciate if Frontier show little more respect to players willing to report a bugs despite actual state of IT ... and decided to finally do something, to make player a feel that reporting a bug really have a sense. And here I really do not mean manual work of Sally and hers enormous effort to be helpfull in this (not only) regard.

 

[HappyMoonMonkey]

Yes, better bug tracking. No automatic closing of tickets with version updates that don't fix items. Better association of new tickets to old etc.

 

[Ian Doncaster]

Given the amount of effort that has clearly gone into making a bug tracker and bug reporting process worse in pretty much every respect than, for example, "installing Bugzilla and ignoring it" I think the best thing that Frontier could do to improve the situation with minimal effort is to shut it down entirely.

If they at some later date wish to receive bug reports again, they can endorse the de facto bug reporting method of "complain on the forums" by opening a dedicated sub-forum. It wouldn't meaningfully change the number of bugs reported or the quality of information available to Frontier about those bugs, but it would be less effort for them and less frustrating for us.

 

[nanite2000]

Given the amount of effort that has clearly gone into making a bug tracker and bug reporting process worse in pretty much every respect than, for example, "installing Bugzilla and ignoring it" I think the best thing that Frontier could do to improve the situation with minimal effort is to shut it down entirely.

I agree they should shut it down, because it's been pretty clear for at least 8 months that they lack the resources to fix even minor game issues, and so keeping the issue tracker running just gives people false expectations and is a waste of everyone's time.

I'm pretty sure not a single bug from the issue tracker has been addressed for about a year now, and that should speak volumes about the game's priority to them. The only bugs they have fixed over the last 12 months are the ones that might have impacted the specific new content they released during that period. No other bugs have been acknowledged, let alone fixed.

The top 20 issues are absolutely a thing of the past now, and I seriously doubt there will be any major bug fixes moving forward.

FDev should take the brave step and announce their future intentions towards the game by either addressing the top 20 issues again, or removing ED from their Issue Tracker altogether.

In the absence of any verbal confirmation either way, we must look at their actions (or inaction in the case of Odyssey).

 

[Pville_Piper]

I stopped bothering with the bug reports because it seemed like nothing was being done. New bugs appear, old bugs re-appear... Silence.

I even stopped suggestions, sorry, but it feels like nobody listens. I post an idea in the forums some white knight comes along to tell me how the game is currently played completely ignoring that I'm suggesting improvements. So many parts are still bare bones after so many years... I try to enjoy the game as it is but the bugs and lack of deeper game play just get in the way. I really don't know how long the invasion saga will go on or how long I will continue to play the game.

 

[CMDR Karrde Sun]

If you've been bug watching elite, frontiers behavior has always been surprising in what they both leave in and fix. After the initial chances frontier had to prove they were using the bug tracker as an actual bug tracker failed, i just concluded they weren't.

My guess at reality: Frontier have their own qa with input from support, crash logs etc. They have their own bug fix cycles that work exclusively on their internal testing. However, communities need to be pacified.. and frontier's favourite pr win is "listening to the community", so decided that if there is any spare time ontop of new features and their own bug fixes, they'll fix the most valuable issues to the community and claim the pr goodwill.

Hence they built a popular vote system not a bug reporting tool. Its pr info.

Its not the same personal narrative of being managed along in positivity bliss, but the above checks out to what they've actually done? Its easier to believe something with less problems with the theory? Basic science.

EDIT: Sorry to the topic, my guess is you need to change how frontier triage bugs they know about. Managers and QA meet periodically during development to discuss known bugs and what gets fixed. During this triage is where they implement business decisions on quality.. and frontier have set their bar too low. Ie, they'll only fix critical or severe bugs. Even major ones are getting set to "will not fix" (mining pulse not working, carrier commodity markets not delivering cargo until second visit, planets not appearing in supercruise).

 

[Rhaazerus Orzhov]

Thanks for sharing @rootsrat
This should be pinned in the forum.

At first use of the bug tracker I was naively under the impression that it was an actively used tool.
After seeing multiple issues I have reported as well as others from players with similar issues, all either expiring or being marked as "fixed" while remaining in the game;
I came to understand how under-utilized and defunct the issue tracker has become.
And I absolutely agree, Elite Dangerous needs some attention and refocus on long-standing issues and improvements more than it needs new content of any kind.

We have been nursing this thread for a full year now, specifically geared towards issues that are in the expiration bin, let alone the plethora of new reports.

Thread for publicising your issues that are otherwise expiring due to lack of votes/confirmation

Unfortunately, rather than deciding on priority and investigating bugs themselves, FD have outsourced the prioritization and confirming of bugs to the players, turning it into a popularity contest. This is the game we have to play now if we want FD to pay attention to the issues that are not...

forums.frontier.co.uk

I know we won't see it happen, but I think what would need to happen is for the entire issue tracker to be clean-slated and revamped and staffed with employees to actively handle the issue tracker.
But like tearing down a house to rebuild the foundation, no one ever wants to do what it really takes to fully fix the problem until they have to.

 

[angusmadmac1]

All I can say is the bug tracker is working in Fdevs eyes ? The whole idea seems to put off people from reporting issues which I would say it's successfully working ? This means the Devs can turn round to big chief and say look we are getting better the bugs reported have dropped, steadily since we "improved " it. If you have proof and you document it , you can't just expire it because it didn't get 10 votes ? Or I can't vote on this bug cos I voted on those other bugs ... I gave up they won .

 

[Maxx Thorogood]

The game has been in "as is" mode for a very long time.

 

[St0rmFury]

Incidentally, how fast are bugs resolved in the issue tracker for other Frontier games?

 

[Ian Doncaster]

My guess at reality: Frontier have their own qa with input from support, crash logs etc. They have their own bug fix cycles that work exclusively on their internal testing.

Agreed. They fix plenty of bugs - including some with pretty trivial severity, presumably if the fix is also trivial - but the public issue tracker appears to have no meaningful connection to what they fix (which may partly be because they often forget to mark fixed bugs as being fixed there, and even when they do it lacks the "notify the reporter" feature that every other bug tracker in existence has)

Incidentally, how fast are bugs resolved in the issue tracker for other Frontier games?

Hard to say. For the games that they develop as well as publish, and only putting as much effort into identifying duplicate or invalid reports as Frontier do:

Resolution rate is Fixed+By Design+Invalid+Duplicated / Reported
Fake Fix Rate is assuming that all bugs which are not Confirmed are the collective delusions of only 10 Elite Dangerous players or 3 Other Game players.

Game	Reported	Currently Confirmed	Fixed	By Design, Invalid or Duplicated	Fix rate	Resolution Rate	Fake Fix Rate
Elite Dangerous (excluding separate Alpha/Beta reports)	38649	227	1288	5033	3%	16%	85%
Planet Coaster (PC/Mac + Console)	435 + 353	72 + 8	45 + 43	31 + 108	11%	29%	52%
Jurassic World Evolution (1+2)	642 + 1390	15+80	57 + 147	208 + 432	10%	42%	68%
Planet Zoo	8353	85	163	663	2%	10%	66%
F1 Manager	1469	14	100	392	7%	33%	87%
All self-developed games except Elite Dangerous	12642	274	555	1834	4%	19%	67%

So ... their fix and resolution rate for Elite Dangerous isn't that much less than their fix and resolution rates for the average of their other games, but in absolute terms they've fixed and resolved over twice as many ED bugs as the rest of their games combined. ED is of course older than any of the other games - though not older than the combined age of all their other games - which will skew this a bit. (Is it just more full of bugs than the average Frontier product? Well, again, it's also bigger than the others and I can't see a way to tell if it has more "bugs per square feature" or whatever you want to measure)

On the "fake fix rate", Elite Dangerous is considerably better than almost all their other games (F1 probably benefiting from being new and still in the "early patches" stage) - if a bug does fight through the obstacles to get to Confirmed, it has a good chance of getting fixed. Of course, if they allowed ED bugs to be confirmed with 3 confirmations rather than 10, like the rest of their games, the number of Confirmed bugs would rise substantially and the fake fix rate would probably fall.

On that basis I'd say (and I'm surprised by this first point):
- report bugs to the bug tracker if you have the organisation (or alts) to confirm them without much effort; despite Frontier's best efforts to portray it as pointless Confirmed reports do actually seem to do some good over and above forum complaints
- ignore the voting system [1], the vast majority of those fixed bugs will never have got any significant vote count, it's not necessary

[1] As an aside on this, if you open the issue tracker for all games, then sort the Confirmed bugs by Top Voted, the first non-Elite bug is a Planet Zoo crash bug at 29th (and you have to get to 108th before a third game appears in the list). Elite Dangerous bugs get into the "only 1 bar of votes" stage by about 40th.

 

[zimms]

I'm afraid there's nothing that can be realistically done about it. Frontier is a company - and we have a pretty good idea how Elite has performed over the last couple of years. The result is that the size of the team working on Elite is nowhere near where "we" would like it to be. The current pace of development (and bug fixes) is all that's possible.

On the issue of reintroducing old bugs time and time again: That is something Frontier can/should/must improve upon. That's not down to manpower.

 

[space voyager]

I'm afraid there's nothing that can be realistically done about it. Frontier is a company - and we have a pretty good idea how Elite has performed over the last couple of years. The result is that the size of the team working on Elite is nowhere near as "we" would like it to be. The current pace of development (and bug fixes) is all that's possible.

On the issue of reintroducing old bugs time and time again: That is something Frontier can/should/must improve upon. That's not down to manpower.

True. Fixing bugs does not generate revenue. New content (or livery) does, so FDev focuses on it. But man, those bugs are biting hard and worst, they are making new content hardly playable.

Rootsrat, thank you for the thread, I hope FDev at least notices its existence.

 

[Astaran]

well, the Stellaris Dev Team has a pretty interesting approach. They've divided their Dev Team into two teams. One would develop new content and DLC, the other team would go back and fix bugs, rebalance and rework old content. Both teams have their own development cycle and releases.

I really don't understand why "fixing bugs" and "creating new content" need to be mutually exclusive in Elite


Edit&PS:
Yes, kill the Bug&Issue tracker. Burn it on a stake and nuke it from orbit. Replace it with a simple sub-forum for bug reporting

 

[Darkman Returns]

The Issue Tracker has been in Legacy Mode for a long time, though in fairness it's not much of a legacy.

FDev would rather stick with something of their own creation that doesn't work than switch to one of the many alternatives that do. That and the on/off Top 20 Issues and the on/off Transparency on the forums make it seem sometimes that FDev are either disorganised, resource limited, or just unaware of just how much these issues affect their game and/or how to fix them.

The developers they showcase on streams are obviously genuinely excited about the work they are doing and how it is going to be received by the playerbase, so it is truly baffling that the game is being held back from reaching its true potential by bugs and issues that have been reported by so many for so long.

 

[Ned Flandalorian]

The developers they showcase on streams are obviously genuinely excited about the work they are doing and how it is going to be received by the playerbase, so it is truly baffling that the game is being held back from reaching its true potential by bugs and issues that have been reported by so many for so long.

There was also the time that a bug was encountered on a livestream when a dev was present, and he recounted on his next appearance that he immediately went and fixed that bug after the stream. So why are the reported issues from the tracker being ignored so much?

 

[Astaran]

There was also the time that a bug was encountered on a livestream when a dev was present, and he recounted on his next appearance that he immediately went and fixed that bug after the stream. So why are the reported issues from the tracker being ignored so much?

My brother is a code monkey himself. He told me that some bugs are easier to fix than others and sometimes even finding the cause of a bug is a massive and time intensive challenge, let alone finding a way to fix it without breaking even more stuff