The Mistaken Assumption Behind Q&A Sites

The NYT has an article this morning about the rise in Q&A sites, like Quora and Stack Exchange. But in my view, there’s a mistaken assumption behind them: That the answers to most or all of your questions exists out there somewhere on the Net or in the brains of experts,  if only you could ask the right person.

My view is the exact opposite: I believe that the set of known answers is very small compared to the set of possible important questions.  Most of what we really need to know requires sustained systematic study. In other words, finding out important new answers is expensive.

I worry that Q&A sites help teach people that answers are cheap (or free).  And then we won’t be willing to spend the money to get the answers to the important new questions.

I could be wrong. But I know  I can formulate many many important economics-related questions for which there is no good answer because no one has done the  right research, because collecting the data is too expensive.  I assume that any subject matter expert can do the same in their own field.

Comments

  1. Peter Creticos says:

    Mike,

    I have a different take on this. I believe that Q & A sites (or, what we’re developing as a Problem/Solution finder) are useful (in the sense that the problems have been encountered and addressed by someone else) within given communities of practice as they pertain to operational problems. They are probably less useful, even within a given practice, as they pertain to policy since the challenges in recontextualizing the problem and response are formidable. Also, a well-constructed resource should clearly delineate the limitations of known solutions.

    • Mike Mandel says:

      So you think they work in clearly defined knowledge domains?

      • Peter Creticos says:

        If they are used literally, then I believe that they are useful as “working knowledge” tools. If the results are understood as being metaphorical, then the answers may be suggestive of alternative solutions.

  2. W Zadrozny says:

    You write “I can formulate many many important economics-related questions for which there is no good answer because no one has done the right research, because collecting the data is too expensive. ”

    What are they? Is there a list? Could you post a list of a dozen with an explanation why they are important, and what data would be required to answer them.
    It could be an interesting post.

  3. They could actually be more valuable in raising questions than answering them. (I thought they all were students doing their homework. )

  4. Two responses, one is that I don’t see how spreading known answers devalues unknown answers. If anything, it might raise the demand for the latter.🙂 The second response is that of course all the free stuff on the internet today creates an expectation of free in consumers’ minds. News is a perfect example of this, rampant music piracy does the same for recorded music. Although with recorded music I don’t think anything can be done about it, news and video and Q&A sites and other online services will charge, using micropayments.😀 But until that’s here, free online content is a double-edged sword. On one edge, it brings people online and gets them using it more, if only to cut costs but they soon see all the other benefits. On the other, you create a vocal minority cult of free, who dumbly say they will never pay for content.

  5. Jacob Davies says:

    If you think of them as question-collecting machines primarily and answer-collecting machines secondarily it may be clearer. Lots of questions have no good answer. Q&A sites collect those. Accumulation of questions (or traffic to a given question page) indicates an interest in obtaining an answer.

    The content-farm sites work in a very similar way (generally by analyzing common Google queries that result in no good answers) but they also produce low-quality “answers” and post them on their site. The resulting info is usually worse than useless, but it’s profitable if you can produce it cheap enough. Presumably pages that produce enough ad revenue might get rewritten and have more time spent on the answers.

    Q&A sites don’t so much do that, and it’s not clear that the accumulated unanswered questions are going to be analyzed (i.e. are available to the public in a form that is useful). But at least the potential exists.

    There are an infinite number of possible questions but they are not evenly distributed. “Who is the President?” is common; “Who is the President of Moldavia?” is not so common; “How can I make my socks wash themselves?” is both unanswerable and very, very uncommon. So the marketplace of question-answering is already quite efficient. Q&A sites fill in some of the blanks where search engines and encyclopedias aren’t quite the right solution.

    My experience with Q&A sites is mostly with those for software, mostly stackoverflow.com, and they are *spectacularly* useful, as in, they are often much *more* useful than the official documentation provided for many programming products (e.g. development environments, libraries, toolkits, platforms), because the questions and answers are those of the people who actually use the products, and not based on imagining what someone *might* want to know; because the questions and answers are not larded with corporate marketing-speak or company-specific jargon; because when the answer is “It doesn’t do that” there is no attempt made to conceal that fact.

    The set of questions about programming that are 1) not addressed in official documentation but 2) have answers that are known to other programmers is *enormous*. Also large is the set of questions that are *almost* some question that has an answer, and another function these sites serve is to reformulate questions in such a way that it becomes clear that an existing answer suffices. (In the process, they also direct search traffic for the misphrased question to the useful answer; like Google’s misspelling-correcting search, they serve to fix the problems with imperfect, human-created searches.)

    • Mike Mandel says:

      I suggest that a site that can answer questions about software is not a useful model to apply to other domains.

    • What about a site for math questions, even advanced ones, or parenting questions? There are sites for every topic, with new ones starting all the time. Obviously software programmers are going to build technical Q&A sites for themselves first, but it has expanded to a lot of other fields already. I don’t really get where you are going with this bizarre notion that people answering each others’ questions online somehow makes them less likely to fund research for unanswered questions. If your quibble is with the free model these sites use, I think you have a marginal point, but people have always shared some expertise with each other for free. The types of questions that have required payment in the past just won’t be answered on these free sites.

  6. Funny how Peter and Jacob gave the same answer, that Q&A sites can be very useful to spread practitioner knowledge within narrow technical disciplines, but Peter’s comment is written in opaque, abstract, generalized verbiage while Jacob uses more concrete and understandable wording, with specific examples.🙂 Reminds me of the classic Orwell quote:

    I am going to translate a passage of good English into modern English of the worst sort. Here is a well-known verse from Ecclesiastes:

    I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.

    Here it is in modern English:

    Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.

    This is a parody, but not a very gross one… It will be seen that I have not made a full translation. The beginning and ending of the sentence follow the original meaning fairly closely, but in the middle the concrete illustrations — race, battle, bread — dissolve into the vague phrases “success or failure in competitive activities.” This had to be so, because no modern writer of the kind I am discussing — no one capable of using phrases like “objective considerations of contemporary phenomena” — would ever tabulate his thoughts in that precise and detailed way. The whole tendency of modern prose is away from concreteness.

  7. A question must be asked before it can be answered.

  8. With sites like Stack Overflow, it’s not just about Q&A but also networking and “karma” currency. Among other things, SO has tried to establish itself as a talent platform where the “talents” can showcase their prowess and potential employers/clients can evaluate their scores and answers “portfolio”.

    On the point of the set of known answers being so small, perhaps the concept of “known answer” is misguided. Often answers emerge in response to a question or posed problem. Depending on the domain and the question/problem posed, the answer may not be factual but of the creative/descriptive type, especially in “real world” subjects (as opposed to formal domains) where there are no unique or canonical solutions. In a sense this can be compared to inventions, which are often enough triggered by a problem, and not the inventor’s drive to exhaust all possible solutions.

  9. On the subject of “cheap or free”, well the answers provided on these terms certainly are. Sure people will get conditioned. But why does everything in the world have to cost money? Perhaps we can just accept that in domains where people are willing to give away answers for free (for whatever reason), you cannot build a commercial business model in providing answers for a fee? Except if you roll performance guarantees, insurance, etc. into your offering – but then it’s not Q&A but commercial consulting.

    OTOH probably nobody expects free financial advice, or advice about other domains where it is understood (rightly or wrongly) that expertise and answers to specific questions involve considerable research or custom effort. In such domains free answers (beyond general and generic advice) will not be available, and there will be opportunity to sell research for a fee.

  10. In regard to answer quality and people being unwilling to pay for quality, this can be observed in all domains. To an extent it is a question of business outlook. If a businessperson (in their right mind) sees good ROI potential and time is of the essence, they will pay up for expertise. At least in the commercial world or where you have to pay for work, a lot of potentially useful work (including research, projects, etc.) isn’t done because the funds are not allocated, either because of lack of revenue, revenue outlook, or other priorities. Any business worth their salt will do a cost/benefit analysis, and when the revenue case (or cost savings if an internal project) cannot be made, the project will not be funded.

    • A similar case is when companies don’t want to pay up for skilled labor. During Y2K/dotcom there were relatively short time windows when money was to be made, and businesses were jumping all over each other to hire anybody who claimed to be able to do this and that. In the primary industries (IT) as well as in the multiplier businesses. These days, not so much, as nobody is waving the big checks.

  11. Prashant P says:

    So maybe there needs to be a website for discussing the issues around the more intractable problems in the world, whilst not promising definitive “answers”.

Trackbacks

  1. […] Do Q&A sites answer questions worth asking?  (Michael Mandel) […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Archives

%d bloggers like this: