FD QA hang your heads in shame

Excalibrates

E
I cannot understand how releases are made with apparently minimal quality assurance being done. Fundamental game mechanics are being broken. Do you guys do any testing AT ALL? If you make a change to the RES spawn mechanics it takes about 5 minutes to determine if they're working as intended. Instead it seems like you just push out a release and then take about 5 attempts to get a particular feature working how you want.

So the repair costs are broken AGAIN and they now need to be fixed AGAIN. Someone obviously made a change to the repair cost calculator; but how did you release it without checking if it works? This is just utterly amateurish. Did you want to start making e.g. Python damage millions to repair, again? If so, why didn't you announce this in the release notes?

So you've prevented shields from recharging when you exist an instance. Why have you also prevented them recharging when I'm docked at a station? I dock, enter hangar, leave the game, come back a day later and my shields are still the same level. Congratutlations, now you force players who take shield damage to actually sit in space waiting for the recharge. Given how quickly NPCs can take down a A6-triple boosted shielded Python, that's now a lot of time spending waiting for shields. That's absolutely no fun, and this is a game, it's supposed to be enjoyable.

So, FD, PLEASE:

1) TEST, TEST and TEST again any changes you make to the game mechanics so that you get it right FIRST TIME
2) Give us some proper, detailed, release notes rather than the cryptic one liners. TELL US what you changed and how it affects the game.
3) Come up with a new way of incentivising beta testers, because the current mechanism isn't working.
4) LISTEN to the beta testers, and FIX the issues they raise before release.
 

Nagual

Nagual
Oddly I was thinking along these lines this morning. I can't help but think that since from the very start FDev have been used to releasing Alpha/Beta releases then fixing the bugs then more beta releases then more fixing, they have got into that bad habit. Now the game has been releases, it is still treated as a massive beta ie release the update and patch like crazy. As a customer I find that logic mental. Every release should be as if it your last, your final touches to it. Keep on pumping out releases before they are stable and they could well be the last.

Customers expect stable games, with no bugs. As a customer I don't care what the excuses are, the excuses are not my problem.
 

RustyLeg

R
Wait, what, there's a QA department?

Seriously though, if I released stuff in the state ED patches come out in, I'd have lost my job long ago. Given you are forced to patch even for playing solo, this isn't very acceptable. It really does feel like they're running with 'it compiles, ship it' some of the time.
 

Engadine

E
Personally I'd rather see fewer releases through out the year but with more content/features when they do come.

Also testing for a week or so clearly isn't long enough, I would have thought a month would suit best. I know that bugs will exist, but not like they currently are, that's evident seeing as we're now on 1.3.4 in what, almost 2 weeks since the drop of 1.3, I'm sure 1.3.5 will be upon us by the weekend.

It just seems to be rush, rush and more rush which seems to correspond with an up coming exhibition or such like.
 

marktay

marktay
I work in commercial software development and have always been bemused at FD's approach to Beta testing. It feels more like a version Early Access programme for backers rather that a serious Beta Test process. As the OP said, its about testing. Once a bug has been identified it not just needs to get fixed but the fix needs to be tested to identify unintended consequences elsewhere in the product. FD seem to rush the beta programme (probably to appease the 'have-nots' on the forum who are clamouring for access) and seem content to then play the patch/hot fix whack-a-mole game for a few weeks after (like the current Integrity/Hull Repair issue). And as the OP says, the Patch Notes are not notes - its just a list.
 

LittleJP

L
The seesaw of bugs getting fixed and reintroduced is utterly ridiculous. Most mod teams are more competent. Heck, I can count on the fingers of one hand the times a mod team, much less a professional studio has done crap like this.
 

Shaamaan

Shaamaan
I'd like to chime in - QA's job is to FIND bugs. Find and ONLY find. They don't fix the bugs. They need time to find the bugs, and then they need time to test the fixes to the bugs...

In my eyes the problem is not that Frontier have no QA but rather their management of the project is, on many levels, wrong. Heck, they use players as cheap QA labor - but they still don't listen to feedback unless it's really really loud (like in the case of the 10% sale penalty - I think this was the only case where Frontier backed out of their decision to implement something due to beta feedback).
 

xyphic

X
Frontier's QA team - and yes, they will have one - have a difficult job, because there's a lot of complexity in a piece of software like this. There are many edge cases that simply won't get exercised until the game is out in the wild, and that's what the Beta test should be for. That being said, on the face of it some of the bugs we're seeing of late appear to be either duplicates of ones that were previously fixed, or so obvious that they should have been spotted earlier.

There's a very good case for changing the Beta cycle from one week to one month. This will increase the volume of testing that goes on, and will give the team more time to fix bugs properly rather than hacking out fixes. The version that's finally released should be the same as the last Beta version; some bugs are getting introduced by late changes between final Beta and release. My suggestion would also be to do the post-release updates with a small (one day) Beta cycle to work out the obvious bugs there.
 

Whiterose

Whiterose
I work in an entirely different sector, pharmaceuticals and medical devices, where QA (or QP) have the final say on shipping of a product or batch. I get the feeling that this is not the same in the software field, whereby release is a managment decision rather than a quality one, and is driven by commercial pressures.
 

Luponator

L
Part of the problem, IMHO, is that there's no medium-plan for what known-good would look like. So, there's bugs which are simple coding bugs, yes. But there's also issues which are bugs and logic errors, but as FD refuse to explain anything in detail about the mechanics of the game, the player base has to second-guess. This, again IMHO, is because FD simply doesn't know where they want to go.
Res site spawning algo- is it a bugged, working as intended, somewhere in the middle? Noone knows as FD won't explain. No I don't mean the obvious bug that spawns only mostly harmless ships. I mean in general.
Repair costs- are they fixed? Not fixed? Rebugged? Noone knows, for sure, as FD don't explain what they should be like. We base the 'fixed/not fixed' on assumptions the player base has gathered rather than on comparing should-be with as-is.
The list goes on.

Without a clear target state there's no long term QA. Combine that with the terrible secrecy around game mechanics (background simulator, anyone? Trade dynamics? etc pp) and you've the recipe for the current state: Bugs, repeat bugs, game mechanic breakdowns and un-announced 'fixes' the player base has to find out about and differentiate bugs from intended, but not communicated, changes.

It's worth checking the video that's posted here where DB's presenting form is criticized. Not because of his performance, that's totally besides the point. But for the presenters before, and after, ED. The games may not be your bag, and not your style, but wow, it's evident that some game shops have a vision, engage with their community and build great games. Yes, with Kickstarter backing, too. It can be done. The current state of ED is not the result of Kickstarter. It's the result of something else entirely.
 

NVNM

N
Whiterose said:
I work in an entirely different sector, pharmaceuticals and medical devices, where QA (or QP) have the final say on shipping of a product or batch. I get the feeling that this is not the same in the software field, whereby release is a managment decision rather than a quality one, and is driven by commercial pressures.
Click to expand...

Badly tested drugs can kill people. Badly tested games at worst annoy people.
 
Last edited by a moderator:

Excalibrates

E
FalconFly said:
I do believe the QA team would likely agree to many of your points.
I'm rather confident that the issue is rooted in Management decisions and not the QA Team pointing the decisionmakers to their known mountain of issues.

QA teams don't really release code into production - those decisions are made at another level. QA at those times may or may not agree those decisions being a good idea - but it's typically not in their hands.

I've seen too many occasions where quite potent and qualified QA teams were overruled by Mangement, Marketing and Sales/Finances.
If Software releases were tied exclusively to meeting set QA standards and -authorization for release... We'd literally live in an entirely different Software world :D
Click to expand...

Fair point - in my old line of work, if the software had bugs there was a chance people could die. So the QA (and Safety) team had the ability to gate a release. It's reasonable to suppose that the FD QA team are raising bugs and they're just going back into the backlog for a sprint - but then management are deciding to release without reviewing the PBI list.

Whatever the processes are around deciding to release, it's apparent it's not working. I think everyone was prepared to tolerate a degree a flux some time after release, but establised game mechanics should really have stabilised by now and shouldn't be getting broken from one patch to the next.

The business model that FD have for Elite:Dangerous requires new players to be constantly attracted to the game. If it gets a rep for being constantly broken from patch to patch then new players simply won't be encouraged to start playing. This game already has a problem with approachability, and FD are making this worse not better.
 
You must log in or register to reply here.
Back
Top Bottom