Appcelerator’s User Survey

Every now and then in the world of market research someone does a survey of their users and then tries to pretend that the results can be projected out to the general universe. It doesn’t happen very often, because most people can see the flaws in that logic with a just a glance. However, it’s happened again and this time the results are being promoted as being descriptive of mobile developers at large. So, let’s see what’s wrong with that.

One, while user surveys are great for finding out what your user base thinks of your programs or products or what they’d like to see changed or added to your product mix, they can’t be used to project what people outside your user base thinks.

Appcelerator makes a platform for creating web based mobile apps and they have versions for iPhone, Android, and have promised future support for Blackberry. So, this limits the survey sample to mobile developers targeting one of the two platforms currently supported. It further limits the sample to web developers most likely wishing to support both platforms, and those who are willing to use the Titanium SDK instead of a more common technology like JQuery. And, of course, it limits the sample to developers who have some interest in the Appcelerator product line (a characteristic which I’m afraid is not universal) and have taken the action of visiting their web site.

Already we can see that huge groups of mobile developers have been excluded from the possible survey sample. If you’re going to eliminate developers working natively on a mobile platforms, developers working on platforms other than iOS and Android, developers devoted exclusively to one platform with no need for Titanium, Windows developers, etc etc – in short, every one except those interested in a specific framework for web development (Titanium) on one of two OS’s then you are surveying a very particular subgroup and you are NOT conducting a survey that can be projected onto the larger universe of mobile developers.

Another problem with this particular survey is that they offered entry into a drawing for an iPad as the incentive to complete the survey. In that way they biased their user survey towards developers who are interested in and who want an iPad. Bad practice, even for a user survey.

I guess we shouldn’t blame Appcelerator for this misleading report – after all they aren’t a market research company and may not know any better, though IDC should. My guess is this is just another marketing gimic to generate leads (they require your phone number for the download). We’ve seen more and more of that from IDC recently and that’s sad for those of us in the industry who take market research seriously.