An Incentive-Aligned Mechanism for Conjoint Analysis Author(s
Transcription
An Incentive-Aligned Mechanism for Conjoint Analysis Author(s
An Incentive-Aligned Mechanism for Conjoint Analysis Author(s): Min Ding Source: Journal of Marketing Research, Vol. 44, No. 2 (May, 2007), pp. 214-223 Published by: American Marketing Association Stable URL: http://www.jstor.org/stable/30162469 Accessed: 17-07-2015 18:13 UTC Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/ info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. American Marketing Association is collaborating with JSTOR to digitize, preserve and extend access to Journal of Marketing Research. http://www.jstor.org This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions MINDING* Thisarticlespecifies,analyzes,and validatesa rigorousand practical The mechanism mechanism(game) forconjoint applications. truth-telling and has truth tellinginconjoint requiresonlyone real productvariation thusmakingit possibleto incentive as its BayesianNash equilibrium, in most Usingthe iPod package conjointapplications. alignparticipants that the mechanism as the context,the authorshows empirically comparedwitha standard substantially improvespurchaseprediction conjoint procedure. forConjoint AnIncentive-Aligned Mechanism Analysis research. of marketing Conjointanalysisis a centerpiece Since the methodologywas introducedto marketing 30 yearsago in a seminalarticleby Green approximately realizhavebeencontinuously and Rao (1971), researchers ing new and majoradvancesin thefield,includinghierarchical Bayesian estimation(Allenby and Ginter1995), polyhedralmethods(Toubia,Hauser,and Simester2004; Toubiaet al. 2003), and partialconjointprofiles(Bradlow, Hu, andHo 2004). Almostwithoutexception,however,conjointdata have thatofferno consebeen collectedin hypothetical settings economicsliterature The decisions. for quences participants' and Hausman Diamond and Camerer 1999; Hogarth (e.g., 1994; List 2001) has longwarnedabouttheperilsof inferbecause in such hypothetical conditions, ringpreferences their true to are not incentive aligned report participants in Grewal, research, Ding, Recently marketing preferences. andLiechty(2005) showedthatdatacollectedin suchhypothandatacolhaveweakerexternal theticalsettings validity lectedfromincentive-aligned Theyfindthatthe participants. thehypothetichoiceconjointoutperforms incentive-aligned cal choiceconjointin out-of-sample predictions. Smeal Collegeof Busiof Marketing, *MinDing is AssistantProfessor ness, PennsylvaniaState University(e-mail: minding @psu.edu). The fromGaryLilienon several comments constructive authoracknowledges previousversionsof thisarticle,as well as thosefromRajdeep Grewal; in presentations givenat GuangGuo; JohnLiechty;Bill Ross; participants andtheMarketofTechnology, Institute Massachusetts CornellUniversity, JMRreviewin Atlanta;and thethreeanonymous ing Scienceconference financialsupportfromtheSmeal Colers.The authoralso acknowledges and assistancefromthe StateUniversity, lege of Business,Pennsylvania State andAuctionsat Pennsylvania forEconomicManagement Laboratory This articlewas processedunderRussellWiner,and William University. Mooreservedas guesteditor. to reader and authordialogue on JMR, visit To read and contribute http:/lwww.marketingpowercom/jmrblog. Association (c)2007,AmericanMarketing ISSN: 0022-2437(print),1547-7193(electronic) it seems criticalthatconjoint In lightof thisliterature, to incorporate properincentivesinto attempt practitioners torevealtheir aremotivated theirstudiesso thatparticipants thistaskis not trivialfor truepreferences. Unfortunately, mostapplications.Standardguidelinesin theexperimental economicsliterature (Smith1976) requirethatparticipants on all tasks,whichimplies be paid fortheirperformance is askedto evaluate a participant thatall productvariations to purchaseor conshouldbe availablefortheparticipant will sumeif he or she so chooses.Anymissingvariations In most the make prodconjointstudyhypothetical. likely have and researchers uct categories,conjointpractitioners access to onlya fewproductvariations, makingit unclear howto incentive usingtheestabalignconjointparticipants and in bothmarketing lishedguidelines.Existingliterature issue. economicsis silenton howto addressthisimportant An ideal solutionto thisproblemshouldsatisfyseveral drivenandprovidegeneral criteria. First,itshouldbe theory guidelinesfor conjointpractice.Second, it should not changesto existingconjointpractice. requiresignificant Third,thesolutionshouldbe applicableto variedproduct stagesby demandingas few categoriesand development variationsof real productsas possible.Fourth,the addishouldbe and participants tionalburdenon practitioners minimal. thatsatismechanism Thisarticledevelopsa truth-telling inforofincomplete fiesthesefourcriteria usingthetheory mechanism mationgames(morespecifically, designtheory; see Mas-Colell,Whinston, foran accessibleintroduction, thatit is in thebest and Green1995). I showtheoretically Nash the (BNE)equilibrium interest-namely, Bayesian in the proposedmechanismto respond of a participant in conjoint.In an empiricalstudyusing iPod truthfully betteroutleads to substantially packages,thismechanism of-sample predictionthan a standard (hypothetical) bias (definedin thathypothetical approach.It also confirms as thebias inducedby thehypotheeconomicsliterature 214 This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions Research JournalofMarketing Vol. XLIV (May 2007), 214-223 An Incentive-Aligned MechanismforConjoint Analysis 215 a real a taskinwhich a participant canpurchase byadding or to on his her inferred willingness pay product, depending is thatan behind thismodification (WTP).Theintuition caninfer a participant's structure experimenter preference the on thebasisofhisorherresponses, which enables to the WTP for variaexperimenter predict participant's any tionoftheproduct Aslongastheparticipant studied. being theidentity comdoesnotknow oftherealproduct before to theconjoint heorshewillbeincentivized study, pleting THEORETICALFRAMEWORK that the toallconjoint toensure respond questions carefully inwhich a Mechanism studies design theory problems willinfer hisorherentire structure experimenter preference wants to obtain certain information about principal private As a it is to result, possible appendconsebutagents willnotreport thisprivate informationaccurately. agents, all to onerealproduct. quences conjoint responses using aregiven theproper unless incentives truthfully they bythe I incorporate theBDM (Becker, and DeGroot, Finally, Theobjective ofa design is toidentify a mecha- Marschak principal. toensure thatitis inthebest 1964) procedure itis in thebestinterest nismforwhich oftheagents to tohavetheinferred interest oftheparticipant WTPequal their true Unlike most theoretireport types. applied game hisorhertrue WTP.TheBDMprocedure, which guarantees in whichtherulesof thegamesaregiven that cal models, itis inthebestinterest of a to state hisor participant is toidentify andthetaskofresearchers the hertrue exogenously hasbeenwidely usedineconomics, WTP, though behavior of suchgames,mechanism (e.g.,equilibrium) introduced intomarketing and (Wertenbroch only recently focuses on the of the and itself, design design game(rules) Skiera Theprocedure involves thefollowing 2002). steps: researchers toidentify a gamestructure suchthat a attempt states hisorherWTPforanitem, (1) A participant (2) a desirable behavior truth in the is embedded (e.g., telling) is drawn from a distriuniform) price randomly (typically, A mechanism contains twoelements: (1) a message game. and(3) theoutcome isdetermined as follows: Ifthe which a player canchoosetosenda message bution, spacefrom drawn is than the stated the WTP, participant higher and(2) anoutcome function that determines theoutcomes price willnotbeabletopurchase iftheprice the but drawn item, foranygiven ofmessages combination sentbyallplayers. islower than or to the stated will the WTP, equal participant When combined with andpayoff a beliefs, functions, types, be able to the item but will the purchase pay only randomly mechanism defines anincomplete information game. drawn As a result, either orunderstating price. overstating Ingeneral, theexisting mechanism literature stud- WTPwill design lead to an inferior outcome for the participant, iestwotopics: socialefficiency 1971;Groves andthe (e.g.,Clarke is to hisorher state participant's optimal strategy maximization 1973;Vickrey 1961)andrevenue (e.g.,Myer- true Inthemechanism WTP. I the herein, proposed replace son1981;Wilson1993).However, noneoftheexisting stated WTPwith inferred WTP. mechanisms canbereadily totheconjoint context applied 1 presents thecomplete mechanism Figure graphically. because to that either are cur- Note pertain products/goods they that areinformed about the entire participants process available orwillbeavailable in rently (e.g.,auction design) before thestart of the task. The mechanism (game) conjoint thenearfuture ifparticipants sochoose (e.g.,public goods). proceeds as follows: eachparticipant the First, completes In addition, theexisting mechanism literature has design standard task as which can use usual, conjoint anytypeof almost restricted itsattention tocasesinwhich exclusively choice, conjoint methodology (e.g., rating, polyhedral). theagent's information is onedimensional, private (type) theexperimenter reveals onerealproduct tothe Second, whereasconjointparticipants' is privateinformation that could Third, participants they potentially purchase. multidimensional. after allthedata,theexperimenter estimates the collecting I specify a new, sound mechanism to Thus, theoretically their preferences (partworths) using conjoint motivate truth inconjoint Theintuition for participants' telling analysis. andinfers eachparticipant's WTPforthereal responses theproposed mechanism comes from truth-telling recasting product. the inferred theBDMproceFourth, WTP, using theconjoint as a gamebetween a participant andthe duredetermines study a whether will be abletopurparticipant From thisperspective, a participant's experimenter. prefer- chasetherealproduct if at what and, so, price.1 encestructure couldbe considered hisorher (partworths) TheBDMprocedure ensures that itisinthebestinterest A is in a continuous multidimentype. participant's type of participant tohavehisorherinferred WTPequaltohis sionalspace,inwhich thedimensions num- oraher equalthetotal true but because of in conjoint errors WTP, (e.g., beroflevelsstudied ina given Inturn, conjoint application. and of estimation, design, quantity quality participants' theparticipant's totheconjoint canbe responses questions theBDMprocedure itself doesnotnecessarily considered hisorherstrategy. Theinformation inthisgame responses), that it in will be the best interest ofa participant guarantee isasymmetric: A participant's isknown tohimorher- to type in The respond (a formal truthfully conjoint. Appendix selfbutnotto theexperimenter. In standard (unincentheexperimenter tivized) conjoint, posesvarious (conjoint) to acquireinformation abouttheparticipant's questions buttheparticipant hasnoincentive toadopt a strategy 'An alternativeincentive-compatible type, procedurecommonlyused in the economicsliterature that is theVickreyauction(Vickrey1961),whichis infeisconsistent with hisorhertype. Whena conjoint is recast as anincomplete infor- riorto BDM in thecurrentcontextforthreereasons:(1) It requiresmultistudy in auctionsagainstotherpeopletend players,(2) peopleparticipating mation the task of game, incentivizing conjoint participants ple to deviatefromtheirtrueWTP,and(3) itsubstantially reducestruth-telling becomes tomodifying thisgamesuchthatthe incentivesforparticipantswho believethattheirWTPs equivalent are smallerthan becomes theBNE.I accomplish this thoseofmostotherparticipants. truth-telling strategy thetical nature ofa task)exists forexpensive durable productsandforfrequently (as purchased inexpensive products intheliterature). Therestofthearticle is organreported I first izedas follows: themechanism andpresent specify I then and theoretical describe theempirical study insights. I conclude discuss theresults. with a general discussion and somefruitful directions forfurther research. This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions JOURNAL OF MARKETING RESEARCH, MAY2007 216 Figure 1 A GRAPHIC REPRESENTATION OF THE TRUTH-TELLING MECHANISM Conjoint Participant Types Preference structure Experimenter Productvariation to be used Mechanism Action Responds to conjointtask Reveals the real productto be used in purchase task Infersa participant'sWTP forthe real producton the basis of his or her responses in conjointtask Actualpurchase price (x) determinedby an incentivecompatiblemechanism (BDM) and inferredWTP Outcome Utility Depends on outcome and types andpractice).As a result, monlyassumedin conjointtheory Theoreml a impliesthattruthtellingis alwaystheunique BNE fortheexisting (It is assumed conjointmethodologies. that conjointmethodologies[estimation]are not biased state their preferences.) when participantstruthfully inFigure1, Theorem 1: Underthemechanism specified Althoughthereis littlereasonto believethatan estimation procedurewill producea smallervariancefora non-truthWTPforthetruth-telling ofinferred (a) Ifthevariance strategy thanforthetruth-telling strategy, practitionstrate- tellingstrategy thanthatfornon-truth-telling is equalto orsmaller in thenotionthatevenif erscan takecomfort lb) (Theorem BNF in is the truth gies, telling conjoint unique actuallyexistfora yetstrategies superiornon-truth-telling are able method and to-be participants developedconjoint WTP forthetruth-telling of theinferred (b) If thevariance structures the discover to them, by represented preference thansomenon-truth-telling is greater strategies, strategy similar to will be these as the BNE as is the in truth strategies non-truth-telling superior long unique telling conjoint thoseoftruepreferences. WTP inferred havean expected non-truth-telling strategies different fromthetrueWTP (forthe thatis substantially seetheAppendix). condition, EMPIRICALIMPLEMENTATION precise The empiricalstudyhas two objectives.The firstis to in conjoint(i.e., all inconsistent strategies Theoretically, to conthe in their are not consistent empiricalevidenceregardingwhetherthe truthprovide responses participants tellingmechanismleads to improvedpredictiveperformjointquestions)shouldhavehighervariancethana consisincentives itaimsto testwhether ance.Furthermore, whereasall consistentstrategies(including tentstrategy, lottery selected of a certain truthtelling)shouldhave the same variance(as is comparticipants percentage randomly (only showsthattruth ofthemechanism) treatment tellingin contheuniqueBNE in thismechanism jointindeedrepresents The mainresultcan be statedas undergeneralconditions. Theorem1: This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions An Incentive-Aligned MechanismforConjointAnalysis 217 receive rewards basedontheir theAppleWebsiterecommended. workeffectively sories that onthe Then, decision) forthemechanism.2 basis of a focusgroupsessionwithtenstudents fromthe Thesecond istofurther ofdif- university, I selecteda subsetof this list for the actual objective explore patterns ferences inpartworths between thehypothetical condition experiments. The subsetconsistsof an iPod (storage-size andan incentive-aligned condition thetruth- variation),a case/holder, (e.g.,under headphones,speakers,car audio, In of their Chinese dinner a and The mechanism). telling study spe- power, warranty. numbersof levelswithineach cialsandsnack andLiechty arealso keptthesameinbothexperiments to allow combos, (2005) attribute Ding,Grewal, establish ininexpensive tworeplicable foodprod- foridenticalexperimental findings designs.The finalattribute space ucts:(1) Hypothetical biasexists, and(2) price is is 223541andincludesfourpricelevels. sensitivity bothsmaller andlessheterogeneous inthestandard Experiment1 employsthe iPod Shuffle,the firstnew (hypoI condition. these in addition to the Apple iPod familyafterthe startof this thetical) investigate findingsproducts from thataredifferent In I food. research 2 uses thenextnew addition addition, inexpensive project.Experiment examine there arereplicable whether biaspat- to theiPod family, theiPod Nano,whichwas launchednine hypothetical ternsassociated withphysical features.3 Taken monthsaftertheiPod Shuffle.Of thephysicalaccessories product thesetwoobjectives call fortwoidentically in the iPod package (case/holders, together, headphones,speakers, inthehypotheticalcar audio,and power),halfof them(twospeakers,twocar (thus, designed experiments anychanges biaspattern thetwoexperiments between forthesamefea- audio,and one power)appearin bothexperiments,4 which turecanbe attributed to the feature itself rather than makes it to examine of differonly possible replicablepatterns to[artificial] totestpredictive design parameters) perform- ence in partworthsassociatedwith these five physical andtruth features. ance;eachhas twoconditions (hypothetical andboth usenewdurable that havea price ValidationTask telling), products ofa fewhundred dollars andshareseveral range physical Thepurpose ofconjoint is to predict a conanalysis features. sumer's real-life decisions. As a result, thebestmetric to Therestofthissection discusses theproducts, validation a new method is to examine whether can it conjoint andfitandpredictive judge task, estimation, design, participants, forchoicedecisions to similar predictions I discuss ofdifference inesti- leadtobetter performance. Finally, patterns those encountered in real life. Because consumers mated between usually thehypothetical andtruth-telling partworths toa largenumber ofoptions when make forpriceandfivecommon they mechanism conditions physical areexposed an actual decision Best carries more purchase features in (e.g., both Buy Unless otherwise present experiments. than 20 different in cameras its retail stores at alldescriptions digital to any both noted, apply experiments. ProductCategory giventime),a good validationtask forconjointmethods shouldnotonlybe wellconstructed butalso includeenough to real-life choicesforthe optionsto makeitcommensurate productcategoryunderstudy.A validationtaskwithonly threeorfouroptionsmaybe tooartificially easyto discriminatetheperformance amongdifferent conjointmethods.5 In thecase of theiPod package,forexample,fourprofiles chosenfroma space of 223541wouldlikelybe so different fromone anotherthatpotentialhypothetical bias probably would not affectwhichof the fouroptionsa participant ranksas themostpreferred. Thatis, smallvalidationtasks arenotappropriate forestablishing theusefulnessof a new suchas thetruth-telling mechanism. Thisguideline method, is followedin theconstruction ofthevalidationtask. After informal discussions with students ata major U.S. where thestudy wasconducted, I selected the university foritsoverall match with the AppleiPodproduct category selection criteria. Mostconsumers several accespurchase sories when atthetimeoftheNano they buythisproduct; andShuffle launches, Apple'sWebsiteevensuggested seven iPodgift sets(Starter, AthTeens, Students, College andGadget eachof letes,Commuters, Travelers, Lovers), which ofoneversion consisted oftheiPodandseveral different accessories the Athletes set contains the 1GB (e.g., iPodShuffle, armband fortheShuffle, andsport case).Borthegift setconcept from I define theproduct rowing Apple, intheempirical as aniPodpackage that consists ofa study Design launched iPodandseveral different accessories. To newly In each experiment, twoconditions wereemployed:one determine thespecific attributes andlevelsforeachattrib- that to thestandard(hypothetical) choiceconcorresponds aninitial listbasedonthecommon accesute,I developed andone forthe The mechanism. joint(control) truth-telling mechanismincludesfiveparts:introduction, truth-telling task(in whichtheidentity of the conjointtask,purchasing 2Inexisting allparticipants receive rewards real productthatcould be purchasedis revealed),external research, incentive-aligned basedontheir orchoice. Inrealconjoint howThe controlconditiondoes performance applications, validitytask,and a briefsurvey. cannot afford toaward toevery ever, practitioners expensive products parnotincludethepurchasing task. Asa result, itiscritical totest thevalidity oflottery reward structicipant. tures. Notethat theobjective hereis nottocompare theeffectiveness of incentives with incentives that reward butrather to lottery every participant examine whether incentives work when itisnotpossible toprovide 4Half oftheaccessories those that comewith aniPod;e.g., lottery (excluding incentives forevery basicearphones) wereintentionally differed toensure sufficient variance participant. 3Ifa physical feature tobeexamined ispresent inatleasttwodifferent between thetwoexperiments. sizeandwarranty arespecific for Storage eachis usedina separate Itis possible totestfor eachiPodversion orNano)andthus products, arenotgoodcandidates for experiment. (Shuffle offeature-specific biaspatterns indifferent contexts examining biaspatterns. possible repetition replicable hypothetical andthuscontribute toempirical inthefuture 5Thisis analogous to measuring A superior generalization (defined by computer performance. Bass[1995, orregularity that overdifferent computer willexcelindemanding taskbutmost p. G7]as "a pattern willnotbenoticerepeats likely inhandling circumstances"). tasks. ablydifferent simple word-processing This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions JOURNAL OF MARKETING RESEARCH, MAY2007 218 containsbothexperimental instructions The introduction of theiPod and theaccessories and a detaileddescription The detaileddescriptions usedin theexperiment. (including pictures),whichare identicalforbothconditionsin each experiment,were reproducedfromApple's Web site. in Instructions forthecontrolconditionmimicinstructions standardchoice conjointstudies,exceptthatparticipants wereaskedto selectone packagefroma listat theend of the study(externalvaliditytask) and a randomlyselected wouldbe givenhis winner(fromevery40-50 participants) or her chosenoption(whichincludesnotpurchasingany betweena certain packagefromthelist),plusthedifference 1 and $320 in Experiamountofcash ($250 in Experiment in the ment2) and thepriceof thatoption.All participants controlreceived$10. conditionweretoldthat in thetruth-telling Participants theirresponsesin theconjointtaskwouldbe used to infer theirWTP fora specificproductX, thattheywouldknow theconjointtask,andthat ofX aftercompleting theidentity they would participatein an externalvaliditytask, as received$10, and a All participants describedpreviously. wouldbe randomly winner(fromevery40-50 participants) For each winner,a selectedat theend of each experiment. whether he or she wouldreceive cointosswoulddetermine WTP usingtheBDM proceX on thebasis of theinferred dureor theoptionchosenin theexternalvaliditytask.If X werechosen,a price(x) wouldbe drawnfroma uniform thatincludesall reasonablevaluationsforan distribution iPod package.If x wereless thanor equal to theinferred WTP,thewinnerwouldreceiveX atpricex, plusthedifferencebetweena certainamountofcash ($250 in Experiment he or she 1 and $320 in Experiment 2) and x. In contrast, wouldreceivethecash and no X if x werehigherthanthe WTP. Finally,ifthecoin tossresultedin theexterinferred nal validitytask, the winnerwould receive the option selectedin the externalvaliditytask,plus the difference betweenthecash andthepriceof thatoption. threeparts(theconfortheremaining The instructions joint task,the purchasingtask, and the externalvalidity andfollowthepracticein thefield; task)arestraightforward conditionwereurgedto "image in the control participants thatyou wereaskedto choose RIGHT HERE and RIGHT NOW" in the conjointtask.The productvariations(profiles) used in these threepartswere generatedby SAS designmacrosto ensuredesignobjectivity, experimental whichindicatesthata 72-profile designis themostefficient space (223541).Then,SAS was used designfortheattribute with20 additional 72 profiles, togenerate nonduplicate proand externalvaliditytasks.The 72 filesforthepurchasing profilesweredividedinto24 groupsby meansof therandom sequencegeneratedby SAS (witha few rearrangementsto ensurethattherewas no dominant profilein any givengroup),and thegroupswereused as the 24 choice was addedto each tasks(afteran optionof notpurchasing group)in theconjointtask.In each choice task,a particioptionout pant'sjob was to pickhis or hermostpreferred the of four(threedifferent not-purchasing packagesplus 4 profilesthatwould option).Of the20 additionalprofiles, iPod at thelowest dominate(packageswiththehigher-end iPod price)or be dominated(packageswiththelower-end 16 proThe remaining atthehighestprice)wereeliminated. filesappearedin theexternalvaliditytask,alongwiththe optionof not purchasingany of the 16 profiles.In the was usedas one ofthefoureliminated profiles experiments, taskin thereal product(withoutprice)forthepurchasing thetruth-telling mechanism. Participants fromthe in bothexperiments wererecruited Participants same undergraduate and graduatestudentpopulationat a Experiment1 was conductedone major U.S. university. 2 took monthaftertheiPod Shuffle launch,andExperiment To that Nano launch. ensure one after the iPod month place werepotentialbuyersof thenew product,the participants e-mail and advertisement explicitlystatedthat recruiting in thestudyif theyhad no studentsshouldnotparticipate a digitalmusicplayer.A totalof 49 in purchasing interest 1 and wererandomly in Experiment studentsparticipated mechanism and the to the control (24) truth-telling assigned 2 in Experiment (25); a totalof 117 studentsparticipated and wererandomly assignedto thecontrol(58) and truthtelling mechanism(59). No one participatedin both experiments. Estimation hierarchicalBayesian multinomial A random-effects similarto thatspecified logitmodelis used forestimation,6 and Ginter and Arora,and Gin(1995) Allenby, byAllenby chooses thattheithparticipant ter(1998). The probability fromthetthchoicesetis givenby thejthalternative (1) Pr(zit in thetth wherezitis thechoicemadeby theithparticipant choiceset,ditidescribesthelthoptionin thetthchoiceset and Pi is a vectorof partevaluatedby theithparticipant, A priori,itis assumedthat worthsfortheithparticipant. (2) Normal are that there and vagueconjugatepriorsforB and A. The hierarchical Bayesianapproachmakesit possibleto estimate individual-level parameters((3i), average Eartworth heterogeneity (f3),and thepartworth parameters partworth weremadeafteritwas ensuredthatthecon(A). Inferences vergencepropertiesof the Markov chain Monte Carlo prior analysiswere met.In addition,a rangeof different to valueswas testedto ensurethattheresultswereinvariant for WTP was inferred A participant's thepriorspecification. taskforeachdrawofthe theiPodpackagein thepurchasing and the after averagewas takenas the convergence, sampler finalestimate. Fitand Predictive Performance forExperiment1 appear in The estimatedpartworths Table 1. To assess thein-samplegoodnessof fitforExperiidentiof timesthemodelcorrectly ment1, thepercentage fiedthechoicein eachofthe24 tasksin theconjointexperiThe averagesare mentwas calculatedforeach participant. from 6Thisparticularestimationapproachprecludesan experimenter in the WTP in real time.In the empiricalstudy,participants estimating weretoldthatthewinnerwouldbe invitedbackthe condition truth-telling nextday fortheBDM procedure(if BDM is chosenon thebasis of the coinflip). This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions 219 An Incentive-Aligned MechanismforConjointAnalysis Table 1 PARAMETER ESTIMATES FOR EXPERIMENT 1 Control(Hypothetical) Attribute Me Level Intercept Storage Base: 512 MB 1 GB Mechanism Truth-Telling M Heterogeneityf Heterogeneity 6.19 (.87) 1.41 (1.43) 5.22 (.66) 1.35 (1.35) 3.44 (.43) 2.32 (1.25) 2.86 (.31) 1.13 (.68) Case/holder Base: none Armband forShuffle Sportscase 1.73 (.36) .91 (.28) 1.64 (.87) .58 (.39) .60 (.28) .74 (.25) .90 (.54) .63 (.37) Headphones Base: Applea Apple+ NikeVaporb Apple+ NikeDuroc .72 (.30) .39 (.29) .83 (.48) .39 (.24) .65 (.27) .35 (.28) .61 (.39) .60 (.36) Base: none MonsteriSpeaker Creativespeaker 1.17 (.27) 1.59 (.27) .39 (.24) .41 (.25) 1.56 (.27) 1.87 (.29) .53 (.36) .60 (.36) .72 (.26) 1.89 (.30) .45 (.29) .92 (.60) .24 (.29) 2.01 (.35) .94 (.54) 1.66 (.97) .06 (.25) .17 (.24) .36 (.23) .33 (.19) .38 (.24) .61 (.24) .42 (.26) .46 (.26) Speakers Car audio Base: none Sonycassetteadapter BelkinFM transmitter Power Base: USB USB + battery pack USB + poweradapter Warranty Base: basic Extended Priced .69 (.27) .82 (.43) .07 (.22) .38 (.22) -5.87 (.66) 6.12 (2.83) -5.43 (.47) 1.96 (1.25) aBasicAppleearphonethatcomeswithanyiPod Shuffle purchase. bNikeVaporsportbudheadphones. CNikeDurobehind-the-head headphones. dPricelevelsare $129,$159, $189, and$219. Theyaredividedby 100 beforeestimation forease ofpresentation. meanand standard deviationof 13. ePosterior Posteriormeanand standard deviationofdiagonalofA. Notes:The fivecommonphysicalfeatures arein bold. presentin bothexperiments Table 2 PREDICTIVE PERFORMANCE FOR EXPERIMENT 1 ActualChoiceMatchestheTop PredictedOption Condition Control mechanism Truth-telling TotalNumber 24 25 ActualChoiceMatchesOne of theTopTwoPredictedOptions NumberCorrect Percentage NumberCorrect 4 9 17 36 9 16 identicalbetweenthetwo conditions(78%). However,the predictiveperformancefor the externalvaliditytask forthe truth-telling mechanismin improvesconsiderably 1 (Table 2). Thatis, thechoicesof 36% of the Experiment in thetruth-telling mechanism can be correctly participants predicted, comparedwithonly17% in thecontrolcondition (1 of 17). The (p = .085)7 and 6% witha naiveprediction of in the choices external percentage validitytask that agreeswithone of thetop twopredictedoptionswas also calculatedto measurethesensitivity of thepredictive performance. Withthiscriterion, 64% of choicesin thetruthis used to obtaintheprobability of observinga difference in 7Bootstrap thenumberofcorrectpredictions betweenthecontrolandthetruth-telling mechanism thatwouldbe at leastthisextreme iftheyweredrawnfromthe same population(based on the standardprocedureof resamplingwith The population in thebootstrap is createdbyaggregating the replacement). outcomes(numbersof correctand incorrect from prediction predictions) thecontrolandtruth-telling mechanism undereach situation. Percentage 38 64 tellingmechanismcan be correctly predicted,compared with38% in thecontrolcondition (p = .043). The estimatedpartworths forExperiment2 appear in Table 3. The in-samplefitsare almostidenticalto thosein Experiment1: 78% and 79% for the controland truthThe out-of-sample tellingmechanism, respectively. predictionsare also consistent withthosein Experiment 1 (Table in thetruth-telling 4). The choicesof 34% of participants mechanism can be correctly predicted, comparedwith21% in thecontrolcondition(p = .067). Withthetop two predictedoptions,56% of choicesin thetruth-telling mechanismcan be correctly predicted, comparedwith40% in the controlcondition (p = .047).8 8Todemonstrate theimportance of usinga largeand realisempirically tic validationtaskin benchmarking different (see thedismethodologies cussionunderthe"ValidationTask"subsection)and to providea baseline to existingliterature, wereaskedin Experiment 2 to complete participants This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions 220 JOURNALOF MARKETING RESEARCH,MAY2007 Table3 PARAMETER ESTIMATES FOR EXPERIMENT 2 Control(Hypothetical) Attribute Level Mf Storage Case/holder Headphones Speakers Car audio Power Warranty M Heterogeneityg 9.54 (1.11) Intercept Mechanism Truth-Telling 44.81 (12.2) Heterogeneity 9.56 (.50) 1.34 (.91) Base: 2 GB 4 GB 3.01 (.25) 2.26 (.76) 2.96 (.25) 2.69 (.78) Base: none ArmbandforNano Incaseleatherfolio .83 (.17) .80 (.15) .55 .41 (.26) (.20) 1.18 (.19) .79 (.15) .68 (.28) .35 (.17) Base: Applea Apple+ lanyardb Apple+ Sonye .44 (.15) .57 (.17) .25 .45 (.10) (.19) .51 (.16) .60 (.15) .46 (.19) .38 (.20) Base: none MonsteriSpeaker Creativespeaker 1.00 (.17) 1.40 (.18) .70 .68 (.30) (.28) 1.26 (.21) 1.90 (.19) 1.60 (.56) 1.39 (.51) .31 (.17) 1.43 (.17) .79 .58 (.31) (.26) .32 (.21) 1.56 (.15) 1.22 (.45) .35 (.16) .80 (.16) .79 (.14) .52 .22 (.22) (.09) .79 (.20) .81 (.18) .54 (.21) .33 (.16) .62 (.23) .51 (.15) .63 (.23) 6.85 (1.78) -5.63 (.25) 1.37 (.52) Base: none Sonycassetteadapter BelkinFM transmitter Base: USB USB + Tekkeond USB + poweradapter Base: basic Extended .61 (.16) Pricee -5.21 (.42) aBasicAppleearphonethatcomeswithanyiPod Nano purchase. bAppleNano lanyardheadphones. eSonyFontopiaearphones. dTekkeon foriPod Nano. myPower forease ofpresentation. ePricelevelsare$209, $239, $269, and $299. Theyaredividedby 100 beforeestimation Posteriormeanandstandard deviationof f3. meanand standard gPosterior deviationofdiagonalofA. arein bold. Notes:The fivecommonphysicalfeatures presentin bothexperiments Table4 PREDICTIVE PERFORMANCE FOR EXPERIMENT 2 ActualChoiceMatchesOne of theTopTwoPredictedOptions ActualChoiceMatchestheTop PredictedOption Condition Control mechanism Truth-telling TotalNumber 58 59 NumberCorrect 12 20 in out-of-sample The improvement performpredictive ancesin bothexperiments providesempiricalvalidationfor this mechanism.Furthermore, the proposedtruth-telling a costachieved in is by using improvement performance a practithusremoving incentive effective structure, lottery mechaofthetruth-telling cal hurdleto theimplementation nismforexpensiveproducts. validationtask(fourchoicequestions, an additionaland moretraditional iPod Nano packagesandtheoption each of whichcontainedfourdifferent of not purchasing).Accordingto both the top optionand the top two butthedifled to betterpredictions, mechanism options,thetruth-telling forthetopoption(p = .151) and is onlymarginferenceis notsignificant theimporforthetoptwooptions(p = .088). Thisconfirms allysignificant tanceofusinglargeandrealisticvalidationtasksto comparemethods. Percentage NumberCorrect 21 34 23 33 Percentage 40 56 inPartworths Patterns forPriceandFive ofDifferences Features9 Common Physical under fromthesuperiorpredictive performance Judging it is conin bothexperiments, mechanism thetruth-telling bias indeedexistsforexpensive cluded thathypothetical durablegoods,suchas iPods. This findinggeneralizesthe previousevidencebased on inexpensivefood (Ding, Grewal, and Liechty2005) because iPods and foodrepresent the poles on two key productdimensions(priceand frequencyofpurchase). ofdifferences associatedwithpricesensitivThe patterns by a similarmean ityforiPod packagesare characterized of difference is availableon 9A morerigorousanalysisof thepatterns request. This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions An Incentive-Aligned MechanismforConjointAnalysis 221 "simi- under thetruth-telling mechanism inboth andhigher variance and,asa result, (Theterms mayhave experiments. this in WTPforan iPodpackagethatincludes a higher or"higher" areusedtorefer tothepatterns "lower," lar," orvariance] thehypotheticalspeaker. thevalue[mean under which issimilar orhigher than that under condition to,lower than, Theseresults difthetruth-telling mechanism, respectively.) GENERALDISCUSSION in both ecothosepreviously Research ferfrom reported. onthemechanism thisarticle literature, design Building andHausman nomics 1994;List2001)andmar(Diamond a mechanism that embeds standard specifies truth-telling on andLiechty that, Grewal, 2005)hasshown keting (Ding, in information and studies an conjoint incomplete game inhypothetical conarelesspricesensitive average, people it is the BNE for to reveal their true that participants con- proves todiscount their ditions because theyappear budget inconjoint In addition toitsrigorous studies. inthemean preferences Itis hypothesized that thisdifference straints. several theoretical this mechanism contains foundation, is likely dueto experimental patterns design(e.g.,price desirable will its features that facilitate among adoption whoisnotpaying closeattenintervals used).A participant It not in does require changes existing practitioners. any constraints willlikely small tiontohisorherbudget ignore andcouldbe usedforallofthem conjoint methodologies in andLiechty [2005], (cf.Ding,Grewal, pricedifferences Asa result, a practitioner choice, polyhedral). isonly$1between which theprice difference thetwoclos- (e.g.,rating, or in still could on his her conrely any expertise specific is $2).Inconestprice levelsandthemaximum difference and the same data collection perform joint methodology aremuch lesslikely toignore thediffertrast, participants thismechanism removes Equally important, inthisstudy difference is $90, andanalysis. enceinprice (themaximum the burden of all variations onerous that requiring product is $30).Ding,Grewal, and andtheminimum difference atthetime oftheexperiment (asisrequired by that inhypotheticalbeavailable (2005)alsoreport participants Liechty such that one existing incentive-alignment guidelines), only for conditions havelessheterogeneous sensitivities price isneeded atthetime oftheconjoint variation product study. to theChinese dinner andthesnack combo, contrary special In of nonfinancial terms additional the effort, only major foriPodpackages. whatis observed Further research is itimposes onpractitioners is tocalculate eachpartounderstand howpricesensitivity needed under burden changes WTP for a variation the ticipant's product using conjoint conditions. hypothetical results after the The additional financial burden experiment. Of thefivecommon features be can (which physical is also will to need real limited; practitioners provide prodof differences in partexamined forreplicable patterns uctsas theprizeofrandom butonlytotheextent drawings three Creative and the worths), (Monster iSpeaker, speaker, that theexpected valueoftherandom foreachpardrawing arecharacterized adapter) power bythesametypeofpator is than his her cost(e.g.,a ticipant higher opportunity lowermeansandlowervariances.10 With terns-namely, of a television a 1 in 10 chance of $200 study may require ofthemeanpattern theexception forthepower adapter of whereas a a $2,000 winning, study refrigerator may inExperiment mean in observed 2),allthepatterns (similar ofwinning). this onlya 1 in 100chance Finally, 1 werereplicated inExperiment 2.Thepattern require Experiment mechanism not does additional burdens on impose any parofdifferences associated with theSonycassette is adapter other than toreadextended (but ticipants, needing easy-toa characterized mean and lower variance, by higher except understand) instructions. needsto (Notethata participant thatthedifference inmeandisappeared inExperiment 2. learn if or once he she in different only participates many Thepattern ofdifferences withtheBelkin FM associated studies that contain thismechanism.) conjoint transmitter is characterized similar a mean and lower by Theempirical tests conducted theiPodShuffle and using variance inExperiment 1.Although themeanpattern was Nano iPod the demonstrate external packages superior inExperiment inExperi2, itsvariance replicated pattern ofthetruth-telling mechanism andshowthatsuch 2 contrasts ment with that inExperiment 1.Theempirical validity can be achieved incentives, improvement using lottery evidence heresuggests that ofdifferthepatterns presented reduces thefinancial which costassociated with theimpleencesassociated with features are physical mostly replica- mentation of this mechanism. The also study empirical ble anddepend on thephysical features. It appears that demonstrates the bias exists not (1) following: Hypothetical these arerelated tohowlikely itis that theowner patterns in and cateonly inexpensive frequently purchased product willusea physical Itisconjectured feature, given purchase. butalsoinexpensive durable (2) gories product categories, that under onaverage, conditions, hypothetical participants the ofdifferences associated with patterns price sensitivity tendtounderstate their valuation forphysical features they for durable aredifferent from those for expensive products arelikely touse(e.g.,speakers, thepower andto adapter) and the (3) frequently purchased inexpensive products, patoverstate theirvaluation forphysical features theyare ofdifferences terns inpartworths associated with physical touse(cassette Further isneeded to unlikely adapter). study attributes are feature specific. testthisconjecture. Given itssound theoretical foundation andempirical supOnthebasisofthese itisclearthat the results, empirical this mechanism should lead port, incentive-aligned conjoint WTPfora complex will not be lower product necessarily tobecome field tested andthen tobecome perhaps under an incentive-aligned In thecaseofthe studies condition. standard which will external practice, provide greater fora speaker iPod,a participant mayhavehigher utility areasstilldeserve conHowever, validity. many promising active research. thisgeneral mechanism tinued, First, may wIfformal were tobemade, ratios of between-experiment comparisons not be effective for a box of inexpensive products (e.g., coefficients theprice should beused.However, (e.g.,against sensitivity) cereal that retails for is It to $2.99). unlikelygenerate large theresults would bemuch notlikely inthisstudy, different because the incentives forsomeparticipants totellthetruth in sensitivities aresufficiently similar between theexperiments. I thank enough price ananonymous reviewer forpointing this out. these thepotential isinherently limcases,because penalty This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions JOURNALOF MARKETING RESEARCH,MAY2007 222 inline action issimply toreveal therealproduct valueoftheproduct. itedbythemaximum Second, (type t2).Theoutinwhich a ranthewinner comefunction with theguidelines ofexperimental economics, (o) is theBDMprocedure, of from distribution and a certain domprice(x) is drawn a uniform with amount inthemechanism willbeendowed WTP(wi).Ifxisequaltoorless with theinferred thanthehighest price compared (equaltoorslightly larger money receives butonly willbeusedtopur- than theproduct ofwhich usedina study), themajority wi,theparticipant paysx. is if x the The cannot inthestudy. chasetherealproduct However, participant purchase product greater peoplemay of wi: than wouldinreallifebecause behave thanthey differently toinvestigate D4: The thispotential gain.It wouldbe worthwhile by theoututility eui(--)is determined participant's iftheymust howparticipants behave payfortheproduct o: A1x A2-p Z, hisorhertypet1,andthe comefunction ownmoney. someconjoint withtheir Third, participants experimenter's typet2. butmay understand themechanism not may completely The payoff (utility) eui--)is theexpected participant's offifthey harder tofillit that willbebetter believe try they if the or were abletopurchase in he she increase It utility as possible. outas conscientiously (andas truthfully) x. This drawn at the expected utility themental to investigate wouldbe worthwhile process product randomly price three levelsofexpectabyconsidering ofthelot- couldbe obtained with totheeffectiveness involved. Fourth, regard ofthe over(1) theexperimenter's type t2(theidentity toconduct tions itwould beworthwhile incentive structure, tery the at time of revealed the is not real (2) conjoint), product the effectiverelative and controlled experimentscompare buthaserror dueto is unbiased WTP(estimation that inferred structure with anincentive incentive nessofthelottery ofparorquality andquantity estimation method, isnotfinancially design, thelatter rewards though every participant, of variance x. the characteristics and the ticipants), (3) Using suchasaniPod.Finally, forexpensive feasible products, the valueof continuous distributions, oftheproduct andtheexpected that oneversion mechanism current requires canbe obtained, fora givenparticipant utility to expected As a result, itis notapplicable available. be physically is stated A1: asLemma nophysical forwhich newproduct product which concepts really an This is available. even a (not represents prototype) thatthepurchase LemmaAl: Ifitis assumed price(x) is ranresearch. forfurther andfruitful direction x E [c1, drawn from a uniform distribution, important domly forpractitioners toconduct itis critical fora type In conclusion, utility c2]andw1E [c1,c2],theexpected whochoosesstrategy becauseof their al in theconti participant conjoint applications incentive-aligned is that the It is external jointstudy truth-telling validity. hoped greater todo herein willenable described mechanism practitioners SO. APPENDIX: FORMALSPECIFICATION OF THE MECHANISM TRUTH-TELLING (Al) eui ofwi,respecm andv arethemeanandvariance where theproposed incom- tively, WTP. andWisthetrue ThisAppendix formally specifies itskeytheoretical The mainresultfollowsdirectly fromLemmaAl gameanddescribes pleteinformation Whinif truth-telling unbiased that method is the structure a In line with Mas-Colell, (assumingconjoint properties. informationstrategy isadopted): ston,andGreen(1995)use,theincomplete (D3) (D2) probability, by (D1) types, gameis defined in a conjoint Al: Truth studyis theunique and (D4) payoff(utility)functions. Theorem BNEiftelling mechanism, andonlyif Specifically, has an N1-dimensional D1: Each participant typet1= (t1[1], of t1[2], ti[Ni])E RN1,whereN1is thetotalnumber A specific levelsacrossallattributes. typecorreparticipant Theexperimenter structure. preference spondstoa specific hasanN2-dimensional typet2= (t2[1],t2[2], t2[N2])E A specific ofattributes. RN2,whereN2is thetotalnumber variatoa specific product typecorresponds experimenter tion(profile). T1,T2 denotethesetsofpossibletypesfor Alternaand experimenter, the participant respectively. ofa participant's thetotaldimensions typecouldbe tively, levelsofmorethan as thesumoftheattribute interpreted andtheexperimenter's oneproduct typeis a comcategory, thesecategories. from ofproducts bination D2: Eachplayerknowshisorherowntypebutnottheother distriknowstheprobability type.Theparticipant player's as pi(t2). denoted bution oftheexperimenter's type, sets F = (A1,A2,o--) haspossible D3: Themechanism strategy x ---> o: function an and outcome Z Al A2 (A1,A2) (A2) Equ+ (A3) E{ to chooses ofwiiftheparticipant V is thevariance where intheconjoint tellthetruth study. REFERENCES L. Ginter andJames (1998),"On GregM.,NeerajArora, Allenby, ofDemand," Journal theHeterogeneity Research, ofMarketing 384-89. 35 (August), toDesign L. Ginter andJames (1995),"UsingExtremes Productsand SegmentMarkets,"Journalof Marketing 392-403. 32 (November), Research, andMarketing Generalizations M. (1995),"Empirical Bass,Frank A Personal Science: Science,14(3),G6--G19. View," Marketing Thestrategy (A1)is allpossible Becker,GordonM., MorrisH. DeGroot,andJacobMarschak spacefortheparticipant toall couldprovide theparticipant ofanswers combinations Sequential by a Single-Response (1964),"Measuring Utility his not reflect or which intheconjoint tasks 226-32. 9 (July), Behavioral Science, study, may may Method," EricT.,Ye Hu,andTeckH. Ho (2004),"A Learningorhertrue spacefortheexperimenterBradlow, type ti.Thestrategy andtheexperimenter's BasedModelforImputing realproducts, Conjoint MissingLevelsin Partial (A2)is all possible This content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions An Incentive-Aligned MechanismforConjointAnalysis Journalof Marketing Research,41 (November), Profiles," 369-81. of Colin.F. andRobinM. Hogarth Camerer, (1999),"TheEffects A Reviewand CapitalFinancialIncentives in Experiments: Labor-Production Journalof Riskand UncerFramework," 19(1-3),7-42. tainty, ofPublicGoods," EdwardH. (1971),"Multipart Clarke, Pricing PublicChoice,11(1), 19-33. Peter A. Hausman ValA. andJerry Diamond, (1994),"Contingent Is SomeNumber Better ThanNo Number?" uation: Journal of Economic 8 (4),45-64. Perspectives, andJohn (2005),"IncentiveDing,Min,RajdeepGrewal, Liechty 42 Journal Research, ofMarketing Aligned Conjoint Analysis," 67-82. (February), PaulE. andVithala R. Rao(1971),"Conjoint Measurement Green, forQuantifying Data," Journalof Marketing Judgmental 355-63. 8 (August), Research, in Teams," Theodore Econometrica, Groves, (1973),"Incentives 41 (4),617-31. theHypoA. (2001),"Do Explicit Eliminate List,John Warnings thetical Bias in Elicitation Procedures? EvidencefromField Auctions forSportscards," American Economic 91 (5), Review, 1498-1507. ThePremier Professional for Development Program Researchers Worldwide 223 and Jerry R. Green Mas-Colell,Andreu,MichaelWhinston, (1995),Microeconomic Theory.Oxford:OxfordUniversity Press. Auction MathematMyerson, RogerB. (1981),"Optimal Design," icsofOperations 6 (1),58-73. Research, Economics:Induced Smith,VernonL. (1976), "Experimental ValueTheory," American Economic 66 (2),274-79. Review, R. Hauser, andDuncanI. Simester John Toubia,Olivier, (2004), MethodsforAdaptiveChoice-BasedConjoint "Polyhedral Journalof Marketing Research,41 (February), Analysis," 116-31. DuncanI. Simester, JohnR. Hauser,and Ely Dahan , MarEstimation," (2003),"FastPolyhedral Adaptive Conjoint 22 (3),273-303. Science, keting William(1961), "Counterspeculation, Auctionsand Vickrey, SealedTenders," Journal 16(1),8-37. ofFinance, Competitive KlausandBerndSkiera(2002),"Measuring ConWertenbroch, sumers' toPayatthePointofPurchase," Journal of Willingness 39 (May),228-41. Research, Marketing Robert Oxford: Oxford UniWilson, (1993),Nonlinear Pricing. Press. versity PRINCIPLES OF MARKETING NEW! RESEARCH AnOnline Course Certificate Pharmaceutical basedupontheMarketing RESEARCHisa vigorous PRINCIPLESOF MARKETING self-study training program theprogram allowsyoutoprogress at viatheInternet, Delivered Research CoreBodyof (MRCBOK-(c)). withyourassigned coach-mentor, byconsultations uponrequest. yourownpace,augmented GRADUATE TESTIMONIAL Research course isvery ThePrinciples ofMarketing from anypartoftheworld. beneficial forparticipants andneeds who is a market recommended research career Thecertificate is foranyone pursuing programhighly in the scientific knowledge field. -- Mookda Somsongkul, MANAGING DIRECTOR, RESEARCH MATTERS CO., LTD., BANGKOK, THAILAND course hasbecome theresearch ThePRINCIPLESOF MARKETING RESEARCH profession's highest quality, onthefundamentals ofmarketing research. mostvalued, andmostwidely educational supported program AMAmembers andtheir Theinvestment inthisprogram is$749peremployee Corporate among colleagues. a 12-month discounts are available for 10 or more enrolling during period. group employees MR KETIGR As TRI Ni INTERNATIONAL ENS: AT 117 IMN AMERICAN MARKETING ASSOCIATION The University ofGeorgia 1785 Center Education forContinuing Center Hotel Conference forthecourse,go to www.principlesofmarketingresearch.org orto register Formoreinformation orcall+1-706-542-6692 e-mail [email protected], content downloaded from 132.174.254.159 on Fri, 17 Jul 2015 18:13:30 UTC All use subject to JSTOR Terms and Conditions