Deleted, as my reply has been ninja'd by others.
Last edited:
Strangely enough I did read and I even understood what you posted. I'm not a statistician but, for my sins, I have degree level statistics training as part of my degree so I do know something about statistics, unfortunately, since I'd rather be ignorant of that particular subject.
Are you seriously thinking that using the same IP address is a good indicator of duplication? This is trivial to get around. I could spend a day posting to any forum on the Internet with a unique IP address for each post. I can think of two ways off the top of my head to achieve this. One is to refresh the DHCP certificate on my router and my ISP gives me a new IP address. The other is to make each post from a different Costa, Nero or Starbucks in any big city in Britain. I wouldn't even have to buy a coffee. Heck, I could probably take a walk down Old Street and log onto a different, free WiFi every 20 metres or so.
As for designing and running another survey, I would not even try. Why? You'd never get beyond the design stage since, as you say, there is no way to control or quantify the any of the possible source of error without Frontier's assistance. So, what would be the point? The result would be exactly the same as for your survey. The results could be very good or very bad or anywhere in between but you don't know which it is.
Hello Commanders,
I have just released the results of the Elite: Dangerous Mega Survey. You can find all the links and my comments here: https://www.reddit.com/r/EliteDangerous/comments/6stz53/elite_dangerous_mega_survey_results/
Enjoy!
Next, I want to highlight the limits of this survey. You might think that 12K respondents are not enough to offer a fairly accurate representation of a player-base that is (although we do not know for sure) of several million players. That is actually not true: there is a precise formula to derive how many respondents are needed for a given total population, and even for a population size of 10 Million -- and aiming at a 95% confidence level and at a 5% margin of error -- just about 400 respondents would be sufficient to be statistically significant. So, the number of replies is more than sufficient.
The real problem here is where the respondents came from, since it is there that we can find some selection bias. Frontier (through the moderators) did not allow me to post the survey invite on the official Forum (so if you are reading this on the Forum and this is the first time you hear about this survey, I am really sorry but there’s nothing I could have done) and of course even there the Forum members are only a fraction of the total players. Maybe having the survey sent out through the newsletter would have helped? I doubt that, since I guess that quite a lot of players do not read that either. Short of posting a link to the survey in the login screen of the game itself there is no way to make sure that your sample is perfectly representative of the total population.
So, essentially this should be interpreted as a survey regarding the opinions of Elite players who are also active on Reddit. [A self-proclaimed statistics expert in the comments chastised me, saying that this is a false claim and that even considered all my long explanation of hos this survey has limits, it still has no value whatsoever. Take what follows as little more than a creative exercise then]
The survey was advertised on the Eliteangerous, the EliteOne, and the ElitePS subreddits, and in the corresponding Discord servers (although I also saw it posted on a large FB group: “Elite: Dangerous Community”). Again, even though the total pool of people is much larger than the actual number of respondents (the E
subreddit alone recently broke the 100.000 members record), I don’t think this constitutes much of a statistical problem (the survey has been available for 2 weeks, and I guess this is enough time for a wide enough sample of people to have a chance to see it).
[Note: if FDev would allow it to be advertised on the Forum (or even better in the newsletter!) I’d be willing to re-open the survey for a week or two, to get submissions from non-Reddit users. I doubt that will happen though]
So where’s the issue? Well, the issue is that the average Redditor does not exactly represent the average Elite: Dangerous player (all Elite Redditors are Elite players, not all Elite players are Elite Redditors). But how? Knowing exactly how would help giving this survey more meaning. My guess is: on average Elite players who are also active on Reddit tend to be more “engaged” or “hardcore” than Elite players who are not on Reddit. Why? Because, as we all know, quite a lot of Elite-related stuff happens out of the game (think Thargoid discoveries, news about third party tools, community-organized events, faction politics, and so on). So the “engaged” Elite player (as opposed to a more “casual” one) needs to get online to get more info. Sure, there is the Forum, but my guess is that the 3 subreddits combined get a larger number of hits per day than the Forums do.
What about the issue Forum user vs Reddit user? I know that there is a lot of regulars on the Forum who do not check Reddit (and vice versa). Usually the stereotype is that the population of the Forums is (on average) 1) older and 2) more PvE than PvP oriented. I am not endorsing this assumption, but I think that there might be a kernel of truth there (anecdotally, I know of “older” people who prefer the relatively more familiar format of an internet forum than that of Reddit).
So, although you are free to interpret the data as you see fit, my guess is that considering all of the above we can turn the survey’s limit into an insight, and consider these numbers as meaningful if referring to Elite: Dangerous players who are active online (mostly on Reddit) and therefore:
1) Slightly younger than average
2) More “engaged” than average
If this is true, some of the numbers below are pretty surprising, and (I think) they discredit some misconceptions about the preferences of the Elite player-base. Please keep in mind once again the these are imperfect numbers, not certain scientific results.
OP made quite clear that his survey can't be representative, therefore it doesn't really matter if someone voted twice.
Open is a PvP game mode - why would players seeking a PvP-free environment play in it?
And, they don't like talking.
I think this is it...Did we figure out the definition of majority yet or should I check back after another 12 pages?
Still wrong.Meaningless statistical exercise.
As a survey it's okay but interpreting the results in any meaningful way is not possible given the lack of statistical rigour. Yes, limits were noted but not accounted for.
As I said, I'd be delighted if that had been a good statistical exercise but it wasn't.
Unfortunately.
That's not really the point. It's that there's no way to control large scale duplications as part of an effort by a group or groups to deliberately skew the results. One or two duplicates is one thing. 1200 duplicates as a deliberate skewing is something else.
As said above, representative surveys don't exist in my opinion, you are asking the wrong personThat's not really the point. It's that there's no way to control large scale duplications as part of an effort by a group or groups to deliberately skew the results. One or two duplicates is one thing. 1200 duplicates as a deliberate skewing is something else.
Oh, and what's the point of a non-representative survey?
Note, I deleted my post as it has been ninja'd by others.
Just checking in...
Did we figure out the definition of majority yet or should I check back after another 12 pages?
There is. And there wasn't. Like I said other three times.
There are those who would say that if I had a big sack containing one of every kind of fruit in the world, and then I threw in an extra orange, that the majority of fruits in the sack would be oranges. But then there are those of us who use English properly too.
I play exclusively in Solo, and I am not liked on this board.![]()
As said above, representative surveys don't exist in my opinion, you are asking the wrong person![]()