Publication Bias and the Cross-Section of Stock Returns

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C. Publication Bias and the Cross-Section of Stock Returns Andrew Y. Chen and Tom Zimmermann 2018-033 Please cite this paper as: Chen, Andrew Y., and Tom Zimmermann (2018). “Publication Bias and the Cross-Section of Stock Returns,” Finance and Economics Discussion Series 2018-033. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2018.033. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.

Publication Bias and the Cross-Section of Stock Returns AndrewY.Chen TomZimmermann FederalReserveBoard UniversityofCologne andrew.y.chen@frb.gov tom.zimmermann@uni-koeln.de ∗ May2018 Abstract We develop an estimator for publication bias and apply it to 156 hedge portfolios based on published cross-sectional return predictors. Publicationbiasadjustedreturnsareonly12%smallerthanin-samplereturns.The smallbiascomesfromthedispersionofreturnsacrosspredictors,whichis toolargetobeaccountedforbydata-minednoise. Amongpredictorsthat can survive journal review, a low t-stat hurdle of 1.8 controls for multiple testingusingstatisticsrecommendedbyHarvey,Liu,andZhu(2015). The estimatedbiasistoosmalltoaccountforthedeteriorationinreturnsafter publication,suggestinganimportantroleformispricing. ∗ WethankRebeccaWasykforexcellentscientificprogramingandChrisHollrahforexcellent researchassistance. ThispaperoriginatedfromaconversationwithFrankSchorfheide. Wealso thank Jack Bao, Svetlana Bryzgalova, Stefano Giglio, Yan Liu (AFA discussant), David McLean, Steve Sharpe, Nitish Sinha, Gustavo Suarez, and seminar participants at the Federal Reserve BoardandGeorgetownforhelpfulcomments. Earlierversionsofthispaperreliedondataprovided by Robert Novy-Marx and Mihail Velikov, and we are grateful that they made their data available. Theviewsexpressedhereinarethoseoftheauthorsanddonotnecessarilyreflectthe positionoftheBoardofGovernorsoftheFederalReserveortheFederalReserveSystem.

1. Introduction Thenatureofacademialeadstoanextremelythoroughinvestigationofstock return data. Some argue that, subject to this much questioning, the data will tell you whatever you want to hear. Indeed, the data have informed us of more than one hundred portfolios with high returns and low market risk, leading manytobesuspiciousofinformationobtainedinthismanner(forexample,Lo and MacKinlay (1990), Sullivan, Timmermann, and White (1999), Harvey, Liu, and Zhu (2015), Linnainmaa and Roberts (2016), Chordia, Goyal, and Saretto (2017)).1 Ourinterrogationofthedataissubjecttocontrols,however.Thoughaskilled investigator may be able to coerce her desired answer, the confession is only published if editors and referees deem it trustworthy. Indeed, in reflecting on hisyearsastheeditoroftheJournalofFinance,Harvey(2014)recommendsthat authorsshould“convincethereaderthattherehasbeenminimaldatamining2.” Theeffectivenessofthejournalreviewprocessfindssupportinrecentempirical studies that suggest that stock market anomalies are real (McLean and Pontiff (2016),JacobsandMüller(2016),YanandZheng(2017)). Publication bias is the net result of data mining and the journal review process. Theseeffectsopposeeachother,andthenetresultremainsanopenquestion. In this paper, we propose an estimate of the net result, and apply it to a datasetof156publishedcross-sectionalreturnpredictors. We find that the controlled interrogation of the CRSP tapes is surprisingly effectiveatuncoveringtruecross-sectionalvariationinreturns.Weestimatethat a modest 12% of the typical predictor’s in-sample return is due to publication bias—thatis,whilethetypicalequal-weightedquintilelong-shortreturnisabout 8%peryear,thebias-adjustedreturnis(1−0.12)8%≈7%. This modest bias adjustment comes from the shape of the distribution of publishedt-stats,seeninFigure1. Theleftsideofthisdistributiondisplaysclear evidenceofdatamining,aspredictorswitht-statslessthan2.0areconveniently missing. Therightsideofthisdistribution,however,cannotbeaccountedforbydata 1Throughoutthispaper,“return”refersto“meanreturn.” Wealsoomittheword“mean”in “in-samplemeanreturn,”“truemeanreturn,”etc. 2Data-miningisalsoknownas“data-snooping,”“p-hacking,”“thefile-drawerproblem,”“researcherdegreesoffreedom,”and“theGardenofForkingPaths.” 1

Figure1:Publishedt-statsvsPureDataMining. 80 70 60 50 40 30 20 10 0 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 t-stat srotciderP fo # Published Data Pure Data Mining Model Estimated Model miningalone. Underpuredatamining,t-statsshouldbunchupatthet-stathurdle,andsoweshouldseenotonlyasharpshoulder,butalsoaquickdecay.Tosee this,supposethatthereisnopredictabilityanywhere,andpublishedpredictors are simply those which happened to have t-stats larger than 2. Thenthe t-stats would follow, well, a t-distribution, with many degrees of freedom, truncated around 2. This pure data mining distribution (dashed line) fits the left shoulder of the published data, but it decays far too quickly to account for the right tail. In contrast, our model allows for both data mining and the possibility that thejournalreviewprocessiseffective. Thepowerofjournalreviewisembodied inthedispersionoftruereturns,thatis,theamountoftruevariationinexpected returns. This dispersion can be extracted by fitting the data, and the estimator finds that a large dispersion produces a tight fit (solid line). Intuitively, if true returnsaredispersed,thent-statspickupsomeofthisdispersion,leadingtothe slowdecayofthesolidlineinFigure1. Theestimatedmodelalsoimpliesthatapredictor’sin-samplereturnisinformative about its underlying true return. This result is formalized in a Bayesian expressionrelatedtoJamesandStein(1961)shrinkage,andaveragingacrosspredictorsproducesourheadline12%number. We find this modest bias using two unique data sets. Our primary dataset consists of 156 replicated long-short portfolios drawn from 115 publications in 2

finance,accounting,andeconomicsjournals. Toourknowledge,thisisthemost comprehensivedatasetofcross-sectionalpredictorstodate,3andislargeenough toproduceapreciseestimateofthebiasadjustment.Thebootstrappedstandard errorforour12%adjustmentisjust1.8percentagepoints. Moreover,ourreplicationsperformquitewell: theaveragein-samplereturn is0.72%permonth,withanaveraget-statof4.3. Indeed,wewereabletoreplicate almost every predictor we attempted. Only four predictors that we made serious attempts to replicate produced t-stats less than 1.5. We make our data availableonlineathttp://sites.google.com/site/chenandrewy/code-and-data/. Wefocusonreplicationsinsteadoftheoriginalpublishedstatisticsbecause performancemeasuresareonlypartiallystandardizedacrosspublications. Only abouthalfoftheoriginalpapersreportportfolioreturns, withtheotherhalfreporting only regression results. By replicating portfolios, we ensure that we are feedingcomparableperformancemeasuresintoourestimator. Nevertheless,wesupplementourreplicationswithhand-collecteddata. We searchthepublicationsofour156portfoliosandwritedownanyaverageportfolioreturnsandt-statsthatarereasonablycomparable.Theresulting77portfolios produce a slightly smaller bias adjustment of 10%, though the difference is not statisticallysignificant. It’s important to note that our small bias adjustments apply only to a select group of predictors. As our data consists of predictors published in peerreviewed journals, our estimates are only relevant to predictors that have the possibility of being published. More specifically, our estimated model can be consideredaformaldescriptionofportfolioswithnarrativesandsupplementary resultsthatcansurvivethejournalreviewprocess. Thus,oursmalladjustments donotapplytoportfoliosgeneratedbyuncontrolleddata-miningexperiments, which tend to be dominated by data-mining bias (Chordia, Goyal, and Saretto (2017)). Our model estimates do apply, however, to predictors in the cross-sectional assetpricingliterature. Indeed,ourestimatesleadtotwobroadimplicationsfor thezooofanomalies. Thefirstimplicationisthatthevastmajorityofpublishedpredictorsarenot statistical figments. We follow the multiple-testing literature and construct the 3Hou, Xue, and Zhang (2017)’s dataset of 447 anomalies contains many alternative lagging choicesandvariableswhichwerenotdemonstratedtoproducepredictabilityintheoriginalpapers.Excludingthese,theirdatasetcontains149anomalies. 3

false discovery rate (FDR) implied by our model. We find that the FDR among publishedpredictorsisatiny1.5%.4 Thus,wefindthatthetraditionalt-stathurdle of 1.96 can actually be lowered, and even a t-stat hurdle of 1.79 leads to an FDRof1.0%. Thissurprisingresultmayappeartocontradictmultiple-testinglogic. Ifone runs156traditionalhypothesistests,thenullofnopredictabilitywilllikelyberejectedbypurechance.Doesn’tthislogicimplythatt-stathurdlesmustberaised? Theproblemwiththislogicisthat,whilerunningmanytestsraisesconcerns about lucky rejections, the many tests also provide information unavailable in asingletest. Critically, examiningmanypublishedpredictorstellsusaboutthe nature of the publication process. We find that this process leads to highly dispersedtruereturns,thateacht-statisticisinformativeabouttheunderlyingtrue return,andthusahight-stathurdleisnotrequired.Thislogiccontrastswiththat oflessstructuredmultipletestingcontrols(suchastheBonferroniadjustmentor BenjaminiandHochberg(1995)),whichdonotestimatethedistributionoftrue returns. Instead,theyusethesamenopredictabilityassumptionfromthesingle testsetting,leadingtostrictlyhighert-stathurdles. The second implication of our bias adjustment is that the deterioration in returns after publication cannot be attributed to publication bias. We replicate McLean and Pontiff (2016)’s result that post-publication returns are about 50% smaller than the returns in the original samples. We go beyond McLean and Pontiff, however, in that we produce a precise measurement of the amount of deterioration that is due to publication bias. We find that post-publication deteriorationis25basispointspermonthlargerthanthepublicationbiasadjustment,andwecanrejectwithextremeconfidencethatthereisnonon-statistical deterioration(p-value<0.0001). This second implication is important because it suggests that mispricing playsalargeroleinthetypicalstockreturnanomaly. Withstatisticaleffectsaccountedfor,thedeteriorationinreturnspost-publicationmustbeduetoeither a decline in risk or a reduction in mispricing. The mispricing story has a compellingeconomicexplanation: tradersactonthepublishedmispricing,bidding upunderpricedassetsandavoidingoverpricedones. Risk-basedstories,onthe otherhand,donotprovideaclearprediction. 4ThisFDRomitsthreepublishedpredictorsthatwefailedtoreplicate. Evenassumingthese predictorsarefalsediscoveries,however,resultsinalowFDRof4%. 4

Ourresults,combinedwithacoupleotherrecentpapers,provideacomplete accountingforthereturnsoftheanomalyzoo. Wefindthatthetypicalanomaly return of 8% per year is 12% publication bias. McLean and Pontiff (2016) show thatanother35%ismispricingthatcanbetradedaway. ChenandVelikov(2017) completethestory,showingthatmuchoftheremaining53%canbeaccounted forbytradingcosts. RelatedLiterature OurpaperiscloselyrelatedtoHarvey,Liu,andZhu(2015) (HLZ),whoalsoexaminepublicationbiasincross-sectionalassetpricingusing astructuredapproach. Theyfindthatat-stathurdleinexcessof2.88isrequired toobtainanFDRof1%,faraboveourestimateof1.79. HLZ’s data is substantially different than ours, however. While our dataset contains only variables that predict returns cross-sectionally, HLZ’s dataset is comprisedofassetpricingfactors,broadlydefined. Thus,thetwosetsofresults suggest that there is much more publication bias in factor models and aggregatereturnpredictorsthanincross-sectionalreturnpredictors. Thereareother differenceinmethodology,however,whichmayberesponsibleforourdifferent results, andunfortunately, wecannotprovideadefinitivereconciliation. Inour view,suchareconciliationisanimportantquestionforfutureresearch. Concernsaboutdataminingbiasinstockmarketpredictorsgobackatleast to Jensen and Bennington (1970) (see also Merton (1987), Lo and MacKinlay (1990), Black (1993)). Formal evaluations of data mining include Sullivan, Timmermann, and White (1999), Sullivan, Timmermann, and White (2001), and Chordia, Goyal, and Saretto (2017), who find strong evidence that data mining leadstospuriousinferenceaboutpredictability. Publishing,however,involvesbothdatamining(atleast,collectivedatamining) and the journal review process. To measure the effects of journal review, one needs a body of evidence on the review process, something which was not availableuntiltherecentproliferationofpublishedpredictors. Studiesthattakeadvantageofthisproliferationhaveyettocometoaconsensus. Harvey,Liu,andZhu(2015),LinnainmaaandRoberts(2016),andHou,Xue, andZhang(2017)findthatmostpublishedresultsarefalse,whileGreen,Hand, andZhang(2014),McLeanandPontiff(2016),JacobsandMüller(2016)cometo theoppositeconclusion. Our paper brings to the debate a more structured model. This structure al- 5

lows us to examine both bias-adjusted returns (à la McLean and Pontiff (2016)) andbias-adjustedstatisticalsignificance(àlaHarvey,Liu,andZhu(2015))inthe same framework. Additionally, our paper brings to bear the most comprehensive set of cross-sectional predictors to date, and we make this data publically availableathttp://sites.google.com/site/chenandrewy/code-and-data/. Outside of finance, the literature on publication bias is large (see ChristensenandMiguel(2016)forareview). OurapproachissimilartoHedges(1992) and Andrews and Kasy (2017), who also explicitly model publication bias. ElementsofourbiasadjustmentarealsofoundinEfron(2011)andLiu,Moon,and Schorfheide(2016). Our model complements Liu, Lu, Sun, and Yan (2015)’s model of anomaly discovery. Whiletheirmodelfocusesontradingeffectsandabstractsfrompublicationbias,wedoexactlytheconverse. Thus,thetwomodelscapturetwodistinctcomponentsofthedecayinreturnspost-sample. Otherpapersthatstudy thelong-termdynamicsofanomalyreturnsincludeAltiandTitman(2017)and Penasse(2017). The next section describes a quick 2-page version of our bias adjustment. Section3describesthefullbiasadjustment’smethodology. Themainresultsare inSection4, whichpresentsbiasadjustedreturnsfor156 published predictors. Section5explainswhythebiasadjustmentissmall,andSection6examinesthe implicationsofourestimatesforhypothesistestingandmispricing. 2. A Quick and Dirty Bias Adjustment Thissectionpresentsaquickanddirtybiasadjustmentthatcapturestheintuition and magnitudes of our more rigorous estimation. This quick and dirty adjustmentrequiresonlytwoinputs: (1)thetypicalstandarderroronthelongshortreturnand(2)thecross-sectionaldispersionoflong-shortreturns. Suppose that in-sample returns r are noisy signals of true returns µ , and i i thatµ areonaveragezerobuthavesomedispersion: i r ∼N(µ ,σ) (1) i i µ i ∼N(0,σ µ), (2) whereσisthestandarderrorofr i (thesameforalli),andσ µisthedispersionof 6

truereturns. Onlystatisticallysignificantportfoliosarepublishedandobserved. Thusweobserveportfolioi onlyif r i >2. (3) σ To adjust for publication bias, we compute the expected true return conditionalonpublicationusingBayesrule: µˆ =(1−s)r (4) i i σ2 s= . (5) σ2+σ2 µ The bias-adjusted return is simply the in-sample return r , shrunk at a rate s, i wheres isatransformedsignal-to-noiseratio. Intuitively,ifthepublicationprocessinvolvespurenoise,thestandarderrorσismuchlargerthanthedispersion in true returns σ µ, and shrinkage is 100%. Alternatively, a large σ µ relative to σ implieslittleshrinkage. Tocalculatethebiasadjustment,oneneedsσandσ µ. Fornow,let’sassume that the published average standard error is a good estimate of σ. Using our datasetof156predictors,wehaveσ≈0.18,correspondingtoaportfoliovolatility ofaround3%permonthandasampleofaround20years. σ µ, thedispersionoftruereturns, isnotobserved. However, σ µ isobserved indirectly through the dispersion of in-sample returns. To see this, note Equations(1)-(3)implythatthatpublishedreturnsfollowatruncatednormaldistribution. Thestandarddeviationofatruncatednormalis (cid:113) Std[r i |r i >2σ]= f(σ,σ µ) σ2+σ2 µ (6) where f(σ,σ µ) is an adjustment due to truncation.5 The LHS of equation (6) is directly observed, and everything on the RHS is observed except for σ µ. Thus, Equation(6)canbeusedtoestimateσ µbymethodofmoments. Figure2 illustratestheestimationofσ µ usingEquation(6). Thefigureplots themodel-impliedstandarddeviationofpublishedin-samplereturnsasafunc- 5Theadjustmentis f(σ,σ µ)=1+αφ(α)/(1−Φ(α))−[φ(α)/(1−Φ(α))]2 (cid:113) whereα=2σ/ σ2+σ2 µ,φ()isthestandardnormalpdf,andΦ()isthestandardnormalcdf. 7

tion of σ µ (Equation (6)). A very high σ µ ≈0.90 is required to fit the empirical standard deviation of 0.47% found in our dataset. This high σˆµ implies a very small shrinkage (dotted line) of about 5%. McLean and Pontiff (2016)’s dataset of97predictorsleadstoastandarddeviationof0.40%,andthusalowerσˆµ,but shrinkageisstillbelow10%. Figure 2: Quick and Dirty Bias Adjustment. The solid line (left axis) plots the theoretical standard deviation of published in-sample returns (Equation (6)). “Empirical std dev” is the cross-sectional standard deviation of published insamplereturnsinourdataset. σˆµ (verticaldashed-dotline)isthedispersionof truereturnsimpliedbythedataandEquation(6). Thedashedline(rightaxis)is thetheoreticalshrinkage,definedby Bias-adjustedreturn=(1−[Shrinkage])×[In-SampleReturn] where shrinkage is calculated using Equation (5). “Est shrinkage” is the shrinkageimpliedbythedata. Impliedshrinkageissmall,lessthan10%. Returnsare% per month. The model assumes σ=0.19, the average published standard error inourdatasetof156predictors. 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 1.2 Dispersion of True Returns σ µ dehsilbuP fo .veD .dtS snruteR elpmaS-nI 50 40 30 20 10 0 )enil dehsad ,%( egaknirhS Empirical Std Dev σˆ µ Est Shrinkage The quick-and-dirty estimate overlooks a number of issues. It assumes homoskedasticity,normality,andnopublicationbiasinstandarderrors. Moreover, wehavenotshownthatthissimplemodelprovidesagoodfitforothermoments inthedata. Theseandotherissuesareaddressedinthefullestimationthatfollows. 8

3. Methods: Model, Bias Adjustment, and Data This section describes our methodology. We describe our model (Section 3.1),biasadjustmentestimation(Section3.2),anddata(Sections3.3-3.4). ReaderseagerforresultsmaywishtoskiptoSection4. 3.1. ASimpleModelofPredictorPublication ThemodelissummarizedinTable1. Itisastatisticaldescriptionoftheportfolio publication process. We introduce characters like “academics” and “journals”toclarifytheinterpretation.Therearenodynamics,trading,orstrategicbehavior.Thusthe“truereturn”inthemodelisbestunderstoodasthepublicationbiasadjustedreturn,orthereturninaworldinwhichthepredictorremainsuntouchedbytraders. Table1:ModelSummary Thistablesummarizesthemodel(Section3.1). Allportfoliosthathavearemotepossibility of being published are submitted. Only portfolios with narratives have a chance ofpublication,andtheprobabilityofpublicationp(r /σ |t ,t )isincreasinginthe i i cut slope t-stat. Alldistributionsareindependent. Themodelhas7parameters: µ µ, σ µ, ν µ, µ σ, σ σ,t slope ,andt cut . PropertiesofthePortfolioBasedonNarrativei Truereturn µ i =µ µ +σ µ τ ν µ τ ν ∼student’stwithν µd.o.f µ In-samplereturn r =µ +(cid:178) i i i (cid:178) ∼N(0,σ ) r,i i Logstandarderror logσ i ∼N(µ σ,σ σ) 1 Publicationprobability p(r /σ |t ,t )= i i cut slope 1+exp(t (r /σ −t )) slope i i cut Insearchoftenureorotherglory,academicssearchfinancialmarketdatafor publishablematerial. Asacollective, theacademicssubmiteveryportfoliothat hasaremotepossibilityofbeingpublished. Journals only publish portfolios that meet two requirements. The first requirementisthattheportfoliomustcontaina“narrative,”ordisplayasetofsoft characteristics that meets the journals’ standards. For example, a narrative for momentum is that investors overreact to the past year’s returns. Thus, a narra- 9

tive implicitly places a sign on the portfolio (long past winners and short past losers). Additionally, this narrative implies that returns are generally increasing inthepastyear’sreturns,and,perhaps,itsreturnsarerobusttovarioussubsamplesandportfolioconstructionmethods. Wedonotmeasurethesesoftcharacteristicsdirectly. Instead,wemodelthe quality of narrative i using its unobservable true return µ . The quality of all i narrativesisdescribedbyascaledt-distribution: µ i =µ µ +σ µ τ ν µ (7) τ ν ∼student’stwithν µd.o.f, i.i.d.. (8) µ whereµ µ,σ µ,andν µ areparametersthatgovernthequalityofnarratives. Large ν µ impliesthatµ i isveryclosetoanormaldistributionwithmeanµ µ andstandard deviationσ µ. We allow for small ν µ in order to capture the ideathat there mayberareportfolioswithextremelygoodreturns. Thescaledt-distributionof(7), withitssinglepeak, issomewhatrestrictive. In particular, it implies that there are many distinct signals in the data. For example,analternativemodelmighthavethreepeaks(oneforvalue,size,andmomentum), and then the other predictors are just related variants around those peaks. Wewillseethatsinglepeakmodelisagooddescriptionofthedata. µ µ, σ µ, and τ ν are the net result of authors’ data mining and the journals’ µ narrativescreening,andtheyultimatelydescribewhetherthejournalsarepublishing true returns. Clearly, if µ µ (cid:192)0 the net result is that the narrative screen iseffectiveateliminatingspuriousportfolios. However,ifµ µ =0butσ µ (cid:192)0,the narrative screen is still effective. In this case, though the average narrative produces no returns, some narratives will have truly high expected returns. A low degreesoffreedomν µhassimilareffects. The narrative screen is not perfect. Equation (7) means that some (perhaps many) narratives have µ < 0, and the journals can’t observe µ . Instead, they i i observe the in-sample return r , which is a noisy signal of µ . For a randomly i i selectednarrativei,thein-samplereturnfollows: r =µ +(cid:178) (9) i i i (cid:178) ∼N(0,σ ), i.i.d. (10) i i where we assume that the standard error of the return σ is observed without i 10

error. This assumption can be justified by the fact that the standard error of a typical portfolio’s standard error is two orders of magnitude smaller than the standarderroritself.6 The above assumptions imply that the in-sample mean returns are uncorrelated across accepted portfolios. Theoretically this can be justified because journals are unlikely to accept a new predictor unless it is distinct from previously published ones. Moreover, the empirical pairwise time-series correlation betweenin-samplemonthlyreturnsistypicallysmall. Inoursampleof156predictors,themedianpairwisecorrelationis0.036,and80%ofcorrelationsarebetween -0.36 and 0.43. The full distribution of correlations can be found in AppendixA.3. Narrative portfolios are heterogeneous in standard errors, and standard errorsarelognormal logσ i ∼N(µ σ,σ σ) i.i.d. (11) This assumption implies that standard errors are independent of true returns. One might think that the volatility component of the standard error should be correlated with the true return, as in risk-based theories. The cross-sectional assetpricingliterature,however,isfocusedonportfoliosthatsurviveriskadjustment.Indeed,theliteraturetendstofindawidevarietyofin-samplereturnswith similarvolatilities. Thissettingleadstothejournals’secondrequirementforportfolioi’spublication: itsin-samplereturnandt-statmustmeetasoftthreshold. Thethreshold issoftinthatitisnotastrictcutoff,butaprobabilisticrule:  pub withprob p(r /σ |t ,t ) decision = i i i cut slope (12) i reject , otherwise i wheretheprobabilityofpublicationisgivenbyalogisticfunction 1 p(r /σ |t ,t )= . (13) i i cut slope 1+exp(t (r /σ −t )) slope i i cut This function implies that t is the midpoint of the soft threshold and t is cut slope 6Ifthemonthlyreturnisnormallydistributed,thenthestandarderrorofthesamplevolatility (cid:113) (cid:113) isabout 1 timesthetruevolatility.Asamplesizeof30yearsleadstoafactorof 1 ≈ 2(T−1) 2(600−1) 0.04. 11

Figure3:ModelIllustration.Wesimulatethemodelofbiasedpublication(Table 1),andplotthedistributionsoft-stats. Theleftpanelshowsall“narratives,”that is,allportfolioswhichhavesoftcharacteristicsthatsatisfythejournals’requirements. In-samplet-stats(solidline)arenoisymeasuresofthetruet-stat(dashed line). Thetruet-statisdefinedasthetruereturnµ dividedbythestandarderi ror. Therightpanelshowsnarrativeswhichpassthepublicationthreshold(dotted line). Publication bias is evident in the fact that the published in-sample tstatsarefurtherfromzerothanthetruet-stats. Thesimulationassumesµ µ =0, σ µ =0.30,ν µ =7,µ σ =−1.31,σ σ =0.45,t cut =1.5,t slope =3. 0.25 0.2 0.15 0.1 0.05 0 -6 -3 0 3 6 t-stat ycneuqerF All Narratives 0.25 in-sample truthµ/σ i i 0.2 0.15 0.1 0.05 0 -6 -3 0 3 6 t-stat ycneuqerF 1 0.75 0.5 0.25 0 ytilibaborP noitacilbuP Published Narratives in-sample truthµ/σ i i pubthresh theslope.Theslopecapturesthatfactthatjournalsmakeeditorialdecisionsthat cansoftenastrictstatisticalrequirement. The statistical requirement improves the quality of published portfolios, as r and r /σ are a noisy signals about µ , and thus (cid:69)(µ |pub )>(cid:69)(µ ). Unfortui i i i i i i nately,thecostofthisqualitycontrolisabias: (cid:69)((cid:178) |pub )(cid:54)=0. i i Figure 3 illustrates the model by plotting simulation results. The left panel shows the distribution of all narrative t-stats from a model simulation. The insample t-stats r /σ are more dispersed than their true counterparts µ /σ , as i i i i a result of measurement error (cid:178) . Despite the noise, the average of all narrative i in-samplet-statsisanunbiasedmeasureoftheaveragetruet-stat. Thisunbiasednessismissing,however,frompublishednarrativesintheright panel. Onlynarrativesthatmeetthepublicationthreshold(dottedline)arepublished. Sincenarrativesmusthavelarget-statstobepublished,thispublication biasleadstoabiasinthein-samplet-stats. Inthisparticularsimulation,thetrue t-statsaremuchclosertozerothantheirin-samplecounterparts. 12

Intheformalismthatfollows,it’shelpfultogatherallparametersintoavector θ: θ=[µ µ,σ µ,ν µ,µ σ,σ σ,ν σ,t cut ,t slope ]. (14) 3.2. BiasAdjustmentandEstimation Given parameters θ, Bayesian logic implies a bias adjustment formula. Then estimation of θ by maximum likelihood gives the empirical bias adjustment. Code for both estimation and bias adjustment can be found at http://sites.google.com/site/chenandrewy/code-and-data/. Tounderstandthebiasadjustmentformula,supposeweobserveapublished returnr andstandarderrorσ andwanttoestimatethetruereturnµ .Thenaive i i i ruleassumesin-sample=true: µˆnaive=r . (15) i i Theaboveexpressionisbiasedbecauseitfailstoconditiononthefactthatr is i published—thatis, (cid:69)(µˆnaive|pub )=(cid:69)(µ |pub )+(cid:69)((cid:178) |pub ). (16) i i i i i i (cid:124) (cid:123)(cid:122) (cid:125) (cid:54)=0 Infact,typically,(cid:69)((cid:178) |pub )(cid:192)0,sincethepublicationprocessselectsforportfoi i lioswithlarger (andthuslarge(cid:178) ). i i Tocorrectforpublicationbias,weneedtoconditiononthefactthattheportfolio is published, as well as all other information about µ that is contained in i themodel. Thus,wedefineourestimatorasfollows: µˆ ≡(cid:69)(µ |pub ,r ,σ ;θ). (17) i i i i i Aswehaveacompletemodelofpublication, wecancomputethisexpectation. The simplest way to compute this is to simulate the model, and then plot the averagepublishedµ asafunctionof(r ,σ ).Thisbruteforceapproach,however, i i i resultsinabitofablackbox. Instead, we compute Equation (17) by applying Bayesian reasoning in two steps. The first and key step is to realize that, within the model, the fact that 13

narrative i is published provides no information over and above the model parametersθ.ThisresultcomesfromEquations(7)-(10),whichprovideacomplete descriptionofµ givenθ. Intuitively,publicationmeansthatr isprobablyhigh, i i buttheinformationsetontheRHSimpliesthatwealreadyknowr anyway. Fori mally,thisreasoningmeanswecansimplifyourestimator: µˆ(r ,σ ,θ)≡(cid:69)(µ |pub ,r ,σ ;θ)=(cid:69)(µ |r ,σ ;θ). (18) i i i i i i i i i Thisresultmaybesurprising,andindeed,hassometimesbeencalledaparadox (Dawid(1994),Senn(2008)).7 Finally, we can write down an expression for our publication bias-adjusted return. To do this, rewrite the RHS of equation (18), using the definition of expectationandBayesformula: (cid:90) ∞ µˆ(r i ,σ i ,θ)= µ(cid:48) fµ|r,σ(µ(cid:48)|r i ,σ i ,θ) (19) −∞ fµ|r,σ(µ|r i ,σ i ,θ)= (cid:82) d f µ N ˜f ( N r i ( | r µ i , | σ µ˜, i σ )f i τ ) ( f µ τ | ( µ µ˜ µ |µ ,σ µ, µ σ ,ν µ µ ,ν ) µ) (20) where f N (r i |µ i ,σ i )isjustanormalpdfand fτ(µ i |µ µ,σ µ,τ µ)isascaledstudent’s tpdf(Equations(7)and(9)). Theabovebiasadjustmentlacksclosedformsolutions,soweusenumericalintegrationtocomputebothintegrals. Togainsomeintuition, considerthespecialcasethatν µ −→∞—thatis, the truereturnsµ arenormallydistributed. Inthiscase,µˆ canbecalculatedusing i i textbooknormal-normalupdating: µˆ i =(1−s j )r j +s j µ µ (21) wherethe“shrinkage”s is j σ2 s = i . (22) j σ2+σ2 µ i Equations (21) and (22) capture intuitive aspects of publication bias. The negative effect of publication bias comes down to mistaking luck (high (cid:178) ) for true i 7Thereasoningisquitestraightforward,however,inasimplerproblem. Supposex∼N(y,1), x isnotobserved,andweonlyobserve y if y>0. Thenthedensityofx conditionalony being observedisstillN(y,1). Thesameresultholdsif y isonlyobservedwithprobabilityp(y). The shapeofp(y)mayimplythatyislarge,butwealreadyknowyanyway. 14

returns (high µ ). Lucky portfolios have higher in-sample returns, and thus the i adjustmentin(21)increasesinr . Luckyportfoliosstillcontainsignal,however. i Thus,theadjustmentcontainsasignal-to-noiseratio(22),wherethenoiseisthe standard error, and the dispersion of true returns σ µ measures the signal. This signalmeasuresthepositiveeffectofpublicationbias, thatis, thequalityofthe narrativecontrols. Equations(21)and(22)showthatourbiasadjustmentisanextensionofthe celebrated James and Stein (1961) estimator for a vector of means. These, like otherempiricalBayesestimators,improveonthein-samplemeanofagivenobservationbyincorporatinginformationfromotherobservations(Efron(2012)). The bias adjustment (Equations (19) and (20)) assumes that the model parametersθareknown. Forourempiricalapplication,weestimateθusingalarge crosssectionofpredictorsviamaximumlikelihood. Theinvariancepropertyof maximum likelihood implies that plugging our estimate θˆ into Equations (19) and(20)givesthemaximumlikelihoodbias-adjustedreturn. Maximumlikelihoodrunsintoafewtechnicaldifficulties. Themostimportantisthatestimationofµ µ canleadtobadbehaviorfornumericaloptimizers, asthelikelihoodtendstobecomeveryflatfornegativevaluesofµ µ. Thisproblemisintuitive: Sinceonlypositivereturnsarepublished,thepublishedreturns haveverylittleinformationaboutthetruemeanifthetruemeanisnegative. Thus,wedonotestimateµ µandinsteadsetittothemostconservativevalue of 0. Nevertheless, we find that setting µ µ =0 appears to result in a local maximum (Figure A.2), and, moreover, assuming other values for µ µ has little effect onourestimatedshrinkage(TableA.2). Fordetailsonhowthelikelihoodiscalculatedandoptimizationperformed,seeAppendixA.2. The effectiveness of our estimator is illustrated in Figure 4, which plots bias adjustments computed from applying maximum likelihood to simulated data. As the data is simulated, we can directly observe publication bias. This bias is manifested by the fact that the naive prediction that assumes in-sample = true (solidline)istypicallyhigherthanthetruereturns(bluex’s). Moreover,thisbias increasesinthein-samplereturn,aswellasthestandarderror(leftvsrightpanels). Thefigureshowsthatourestimatoreffectivelyremovespublicationbiasfrom simulateddata.Thebiasadjustedpredictionsrunrightthroughthecenterofthe clouds of true returns, and adjust effectively regardless of the in-sample return 15

Figure 4: Bias Adjustment Illustration. We simulate 400 published portfolios, runamaximumlikelihoodestimationonthesimulateddata,andapplythebias adjustment (Equations (19)-(20)). Portfolios are separated into those below the median standard error (left panel) and those above the median (right panel). The naive prediction (solid line) assumes the in-sample return is equal to the expectedtruereturn. Thispredictionisbiasedupwardcomparedtothetruereturns(x’s)duetopublicationbias, andthisbiasincreasesinstandarderrorand thein-samplereturn. Thebiasadjustedpredictions(triangles)effectivelyadjust forpublicationbias. ParametersareintheFigure3caption. 2 1.6 1.2 0.8 0.4 0 -0.4 0 0.4 0.8 1.2 1.6 In-Sample Return nruteR eurT / detciderP Low Standard Error Ports 2 Naive Prediction True Return 1.6 Bias-Adjusted Prediction 1.2 0.8 0.4 0 -0.4 0 0.4 0.8 1.2 1.6 In-Sample Return nruteR eurT / detciderP High Standard Error Ports 16

andstandarderroroftheportfolio. 3.3. Replicationsof156Cross-SectionalReturnPredictors Ideally, our data should (1) cover a comprehensive set of predictors, (2) use standardizedperformancemeasures, and(3)usestatisticsreportedintheoriginal publications. Achieving all three goals is impossible, however, as performancemeasuresareonlypartiallystandardizedacrosspublications. Onlyabout half of the predictors we examine report portfolio returns, with the other half reporting regression results. Moreover, the published portfolios use a variety of constructions and often only report returns adjusted for factor exposures or characteristics. Thus, we construct standardized performance measures for 156 long-short portfolios by replicating 115 publications in accounting, economics, and finance journals. To our knowledge, this data is the most comprehensive set of cross-sectional predictors to date. The Hou, Xue, and Zhang (2017) dataset of 447 anomalies consists of only 149 anomalies if one excludes alternative lagging choices and anomalies that were not demonstrated to produce predictability in the original papers. The complete predictor list and detailed definitions can be found in Table A.1. We also make our dataset available at http://sites.google.com/site/chenandrewy/code-and-data/. We cannot perfectly replicate all 156 predictors. Our replications have the more modest goals of (1) capturing the spirit of the paper and (2) producing tstatistics above 1.5. We use the cutoff of 1.5 instead of the traditional cutoff of 1.96 to allow for differences in t-stat calculations and updates to data sources. For example, Ritter (1991)’s study of long-term IPO underperformance finds tstats in excess of 5.0 using event time returns. To compare across predictors, however, we must use calendar time, and our calendar time version of the IPO predictorproducesamoremodestt-statof1.6. We achieved both goals for almost every predictor that we made serious attemptstoreplicate. Inonlyfourcaseswefailedtogeneratet-statsabove1.5. For three of these cases we assume the error was ours and omit the predictor. The remainingcaseiscashflowyieldvariance,whichisshowntobestatisticallysignificantinHaugenandBaker(1996)’sregressionresults. Ourportfoliosortsusingcashflowyieldvarianceproducealowt-statof1.20,butwekeepitbecause the returns are monotonic and the long-short return is a respectable 0.39% per 17

month. Panel A of Table 2 describes the types of predictors in our dataset. Our predictors come from top tier academic journals and are quite diverse in the data sourcesused. About 90% of our predictors are published in the “top 3” finance journals, “top3”accountingjournals, orthe“top5”generalinteresteconomicsjournals. The remaining 22 are also published in reputable journals, and include importantpredictorslikeTitman,Wei,andXie(2004)’sinvestmentanomalyandAmihud(2002)’silliquiditymeasure. The predictors use a wide variety of data sources. As one might expect, the majorityuseonlyaccountingdata(62)orusemarketpricesinsomefashion(48). Butanequallylargenumber(46)usediversedatathatincludeanalystforecasts, trading-related measures, and corporate events. Many predictors are valuation measures (13), but they represent only 8% of our 156 predictors. Similarly, 9 of our156predictorsarerelatedtomomentum. Weincludemultiplevaluationand momentummeasuresastheywereconsidereddistinctenoughtobepublished. The“top3”financejournalsappeartotakeinterestindiversity. Allsixofour predictor categories are well-represented in the “top 3” finance row of Panel A. Incontrast, theaccountingjournalsarefocusedonaccountingonlypredictors. This stark change in focus between accounting and finance journals does not affectourmainresults,however. Wewillseethatourmainresultsholdifweonly usepredictorsfromthe“top3”financejournals. Toevaluatepredictorperformance,wecreatelong-shortportfoliosandmeasuremonthlyreturns.Formost(130)predictors,weuselong-shortextremequintilesofthepredictor. Bothquintilesanddecilesarecommonlyusedintheliterature,butusingquintilesreducesthenoiseinmeanreturnsandmakesforeasy comparisonwithMcLeanandPontiff(2016). Theremaining26portfoliosgolongaparticularindicatorandshortanother set of stocks. These portfolios are mostly constructed from discrete predictors, such as Hong and Kacperczyk (2009)’s sin stock (stocks involved with alcohol, tobacco, or gaming) predictor or Cusatis, Miles, and Woolridge (1993) spinoff event predictor. We will see that these indicator variables perform similarly to ourquintilesorts. All but two portfolios are equal-weighted, as most of the original publications focus on either equal-weighted portfolios or Fama-Macbeth regressions. 18

The two exceptions are idiosyncratic volatility (Ang, Hodrick, Xing, and Zhang (2006))andtheGompers,Ishii,andMetrick(2003)governanceindex. Wevalueweightthesetwopredictors,astheyperformfarbetterusingvalue-weightingin boththeoriginalpapersandourreplications. Panel B of Table 2 shows the mean returns and t-stats from these portfolios using the papers’ original in-sample periods. The returns in the original samplesarearound0.70%permonthonaverage,somewhathigherthanMcLeanand Pontiff(2016)’saveragein-samplereturnof0.58%. Toputtheseaveragesinperspective,ourequal-weightedversionsofvalueandmomentumaresignificantly more anomalous than the average predictor, earning 1.01% and 1.24% in their originalsamples,respectively. Value and momentum are not the only predictors with especially high returns. The cross-predictor standard deviation of about 0.50% is nearly as large asthemean,andsimilartothe0.40%standarddeviationinMcLeanandPontiff (2016)’s dataset. Most of this dispersion comes from a long right tail in returns, whichwillbeimportantforidentificationofourbiasadjustment(Section5.3). Thedatashowthatjournalcategoriesproducefairlysimilarreturns. Theaveragetop3financereturnof0.75%is10basispointshigherthantheaveragetop 3 accounting return, but the two average returns are statistically similar given thelargedispersionwithineachcategory.Theaveragetop3financereturnisalso similartotheaverage“otherjournal”return.T-statsdodifferacrossjournalscategories,however.Inparticular,thetop3accountingpredictorshavesignificantly largert-stats, perhapsduetothesmallernoiseinaccountingdatacomparedto marketdata. Incontrasttojournalcategories,datacategoriesproducesignificantdispersioninreturns. Theanalystforecast-basedpredictorshaveanaveragereturnof 1.00% per month, more than twice that of the corporate event predictors. The variation in returns across categories is fairly smooth, however, as for all categories there is another category with a similar return. This smooth variation is consistentwithourassumptionthatallpredictorsaredrawnfromthesamedistribution. Along the same lines, our quintile-sorts and indicator predictors produce verysimilarmeanreturnsof0.73%permonthand0.69%,respectively. Thus,we viewpredictorsfrombothconstructionsasdrawnfromthesamedistributionin ourestimation. 19

3.4. AlternativeData: Hand-CollectedReturnsfor77Predictors Asourmainresultsusereplicateddata, they do notmeasurethe biasinthe originalreportednumbers.Instead,theycaptureamoregeneralconceptofpublication bias. In this concept, each publication suggests a trading strategy and providesavolumeofstatisticsinsupport. Itisthenuptothereadertoturnthe statisticsintopreciseportfolioconstructions,aswedoinourreplications. These replicated expected returns contain bias due to the publication process, which ourestimatorthenmeasures. Ourreplications,however,mayalreadyremovesomepublicationbias. Intuitively, the many authors of the original publications examined more specificationsthanourtwo-personco-authorshipteamdid. Ourlessexhaustivespecificationsearch,then,mayproducelessdata-miningbias.8 To examine this issue, we supplement our replications with hand-collected statistics. Wesearchthroughtheoriginalpublicationsofour156predictorsand writedownanyaverageportfolioreturnsandt-statsthatarereasonablycomparabletoourreplications. Wecollectrawreturnquintilesortswhenavailable,but also collect returns that adjust for factor models and characteristics, as well as portfoliosthatusealternativeportfoliobreakpoints,whennecessary. Table 3 compares this hand collected data with our replications. Panel A shows that our replications are on average faithful to the original papers. The hand-collected portfolio returns average 0.83% per month, just 2 basis points higherthanourreplicationsofthesameportfolios. Similarly,theaveragehandcollected t-stat is 4.57, very close to our average replicated t-stat of 4.60. This similarperformanceisseeninPanelsBandC,whichexaminesubsetsofthedata that depend on the availability of statistics in the original papers. In the panel that examines hand-collected raw return quintile sorts (the same construction we use in our replications), the two datasets deviate in mean returns by just twobasispoints. PanelCshowsthatthehand-collectedrawreturnsareslightly higherthanourreplications,perhapsbecausethatsubsampleofhand-collected datacontainsmanydecilesorts. Overall, thestatisticsforhand-collectedandthematchedreplicatedportfoliosaresimilar. Thisclosematchsuggeststhatourreplicationsandtheoriginal reportedstatisticscontainasimilaramountofpublicationbias. 8WethankStefanoGiglioformakingthispoint. 20

Ultimately,theamountofpublicationbiasinbothdatasetscanbemeasured usingourestimator. WewillseeinSection4.1thatbothleadtosimilarbiasadjustments. 21

Table 2: Summary Statistics for Replications of 156 Cross-Sectional Return Predictors Tabledescribesourreplicateddata. Top3FinanceincludestheJournalofFinance,the JournalofFinancialEconomicsandtheReviewofFinancialStudies. Top3Accounting includestheAccountingReview,theJournalofAccountingResearchandtheJournalof Accounting and Economics. Top 5 Econ includes the Quarterly Journal of Economics andtheJournalofPoliticalEconomy(othereconjournalsdidnothavepredictorsthat wereplicated). AppendixA.1providesacompletelistofpredictors. Monthlyportfolio returnscanbefoundathttp://sites.google.com/site/chenandrewy/code-and-data/. PanelA:PredictorCountsbyJournalandDataCategory Acct Mkt Analyst Trading Event Other Total Only Price Top3Finance 21 37 11 7 10 8 94 Top3Accounting 30 4 0 2 0 0 36 Top5Econ 1 2 0 0 0 1 4 Other 10 5 2 3 2 0 22 Total 62 48 13 12 12 9 156 PanelB:StatisticsforLong-ShortReturnsinOriginalSamplePeriods MeanReturn t-statistic (%permonth) N Mean Std Mean Std Journal Top3Finance 94 0.75 0.47 3.84 2.21 Top3Accounting 36 0.65 0.54 5.20 3.97 Top5Econ 4 0.55 0.16 2.82 2.02 Other 22 0.79 0.39 5.30 3.39 DataCategory AccountingOnly 62 0.65 0.42 4.97 3.32 MarketPrice 48 0.82 0.49 3.81 2.13 CorporateEvent 13 0.41 0.18 2.73 0.92 AnalystForecast 12 1.00 0.52 6.88 4.13 Trading 12 0.65 0.18 2.84 1.03 Other 9 0.92 0.77 3.70 2.85 PortfolioConstruction Quintiles 130 0.73 0.47 4.41 2.97 Indicator 26 0.69 0.48 3.98 2.81 22

Table3:ComparisonofHand-CollectedandReplicatedPortfolios Wehand-collectlong-shortreturnsandt-statsfromoriginalpapers. Wecollectrawreturn quintiles when available, but include adjustments for factor models and characteristics,aswellasalternativesortingmethodswhennecessary. Ourreplicationsareall rawreturnsandquintilesortsorindicatorvariables.PanelAshowsall77hand-collected portfolios,aswellastheourreplicationsofthesame77portfolios. PanelsBkeepsonly portfolios where the original paper provided raw quintile returns. Panel C keeps only portfolios where the original paper provided raw returns. Our replicated statistics are similartotheoriginals. PanelA:AllHandCollectedData MeanReturn t-stat N Mean Std Mean Std HandCollected 77 0.83 0.53 4.57 2.58 Replicated 77 0.81 0.54 4.61 3.11 PanelB:RawReturnsandQuintilesOnly MeanReturn t-stat N Mean Std Mean Std HandCollected 16 0.78 0.69 3.95 1.96 Replicated 16 0.76 0.56 3.69 1.51 PanelC:RawReturnsOnly MeanReturn t-stat N Mean Std Mean Std HandCollected 47 0.90 0.54 4.34 2.25 Replicated 47 0.81 0.54 4.27 2.97 23

4. Main Result: Estimated Publication Bias Adjustments Having described our model, estimation, and data, we are finally in a positiontoshowthemainresults. Thissectionfocusesondescribingtheestimated parameters, bias adjustments, and robustness. Section 5 explains why the bias adjustmentsaresosmall. 4.1. Estimatedparametersandbiasadjustments Table 4 shows the main result: estimated model parameters and implied bias adjustments. We estimate a model of biased publication (Table 1) on our databaseofcross-sectionalpredictors(Table2)bymaximumlikelihood(Section 3.2). The table shows our baseline specification “all,” as well as four alternative specificationsforrobustness. Webeginbydiscussingthebaseline,butallspecificationsleadtosimilarresults. The first estimated parameter shows that many predictors are real. In the baselineestimation,thedispersionoftruereturnsisestimatedtobequitelarge at0.45. Combinedwiththeestimatedfattailparameterof3.89,thisimpliesthat (cid:113) ν thecross-sectionalstandarddeviationofallnarrativetruereturnsisσ µ ν µ − µ 2 = 0.64%permonth. Inotherwords,it’squitecommontofindnarrativeportfolios withtrue,bias-adjustedreturnsof0.64%permonth. Moreover, the dispersion of true returns and the fat tail parameter are preciselyestimated. Indeed,thedispersionoftruereturnsis9standarderrorsfrom zero,showingthatwecan,withlittledoubt,rejectthehypothesisthatallpredictorsarefalse(equivalently,werejectthatallshrinkageis100%). Theremainderoftheparametersdemonstratethattheestimatorworksproperly.Thestandarderrorparametersimplythatthemeanofallnarrativestandard errors is exp(µ σ +0.5σ2 σ )=0.22% per month. This is somewhat higher thanthe meanstandarderrorforpublisheddata(0.19%)permonth,indicatingthatthere is a bit of downward publication bias in standard errors. The dispersion of log standarderrorsissimilartoitsnaivecounterpart,thestandarddeviationofpublished log standard errors. Similarly, the midpointofthe t-stat cutoffisclose to the 1.50 cutoff imposed in our portfolio replications (Section 3.4). The thresholdslopeof11indicatesthatthepublicationthresholdisverysharp,whichcan 24

Table4:EstimationResults Bootstrapstandarderrorsareinparentheses.Weestimateamodelofbiasedpublication (Table1)onmanycross-sectionalstockreturnpredictorportfoliosbymaximumlikelihoodassuming.Shrinkageforaportfolioisdefinedby Bias-adjustedreturn=(1−[Shrinkage])×[In-SampleReturn] where the bias-adjusted return is calculated using Equations (19)-(20). “All” uses our replicationsof156publishedpredictors(Table2). “t-stat>2.0only”usesonlyportfolios with t-stats > 2.0, “top 3 finance only” uses only portfolios from the Journal of Finance, Journal of Financial Economics, and Review of Financial Studies, and “Normal truereturns”assumesν µ =100. “Hand-collecteddata”uses77hand-collectedportfolio statistics(Table3). µ µ =0throughout(seeSection3.2). Thebiasadjustmentissmall,at 12%ofthein-samplereturn. Thissmallshrinkageiswell-estimated,androbustacross allspecifications. ReplicatedData t-stat Top3 Normal Hand All >2.0 Finance True Collected EstimatedParameters Only Only Returns Data σ µ dispersionof 0.45 0.46 0.43 0.66 0.58 truereturns (0.05) (0.05) (0.07) (0.05) (0.10) ν µ fattail(d.o.f.)of 3.89 3.89 3.85 100 6.56 truereturns (1.38) (1.40) (2.55) - (4.63) µ σ meanoflog -1.67 -1.68 -1.54 -1.69 -1.51 standarderror (0.05) (0.05) (0.05) (0.04) (0.09) σ σ stdoflog 0.51 0.50 0.49 0.52 0.68 standarderror (0.02) (0.03) (0.03) (0.03) (0.10) t midpointof 1.61 2.00 1.64 1.56 1.88 cut t-statthreshold (0.08) (0.02) (0.14) (0.05) (0.18) t slopeof 10.97 100 8.31 11.51 6.91 slope t-statthreshold (41.78) - (42.62) (42.35) (37.49) EstimatedBiasAdjustments MeanShrinkage(%) 12.5 10.7 15.1 8.3 9.9 (1.8) (1.5) (3.0) (1.3) (2.5) MedianShrinkage(%) 9.6 8.5 12.5 6.0 7.1 (1.8) (1.6) (3.0) (1.0) (2.6) StdShrinkage(%) 9.5 8.0 10.0 7.0 7.4 (1.1) (1.0) (1.6) (1.0) (1.6) NumberofPredictors 156 132 94 156 77 25

be seen directly in the shape of the published t-stat distribution (Section 5.3). This slope comes with a very large standard error, as slopes larger than 20 are essentiallyverticalanddistinguishingthemisimpossible. Nevertheless,wecan saywithcertaintythattheslopeisfairlysteep,asthe5thpercentileofthebootstrappeddistributionis6.2. The bottom of Table 4 provides the headline bias adjustment number from the abstract. It shows summary statistics for bias adjusted returns, where bias adjustedreturnsarecalculatedbyapplyingBayesrulewithinthecontextofthe model (Equations (19)-(20)). To ease interpretation, we express bias adjusted returnsintermsof“shrinkage,”whichisdefinedas Bias-adjustedreturn=(1−[Shrinkage])×[In-SampleReturn]. (23) The baseline specifications finds that the mean shrinkage is modest, at 12.46%. In other words, for the typical in-sample return of 0.70% per month, thebiasadjustedreturnis0.61%=0.70×(1−0.1246)%). Thissmallshrinkageis well-identified,withtheastandarderrorofjust1.8percentagepoints. Thesemeanshrinkagenumberssummarizeourmainresult.Wecansaywith confidence that the net effect of publication bias on the cross-sectional return predictionliteratureissmall. 4.2. AlternativeSpecifications Our headline result is robust. Table 4 shows that the mean shrinkage is around 12% regardless of the specification. Further robustness is seen in even morespecificationsinAppendixA.3. ThefirstalternativespecificationinTable4limitsthedatatopredictorswith t-stats > 2.0. This specification addresses the concern that the predictors with smallt-statsarenotcomparabletotheotherpredictors. Thisconcerndoesnot affectourmainresult,asestimatingonthe132portfolioswitht-stats>2.0results inaslightlysmallershrinkageof10.7%. One can argue that our measure of publication bias is muddled as it mixes togetherjournalsfromdifferentfieldsandofvariousreputations. The“top3financeonly”specificationshowsthatourmainresultsholdifwerestrictourpredictors to the 94 published in the Journal of Finance, Journal of Financial Economics, and the Review of Financial Studies. According to our estimator, the 26

top 3 finance journals have a publication bias of 15.1%, slightly larger than our baseline estimate, but within one standard error. Indeed, the standard error is noticeablylargerforthetop3financeestimate,asthenumberofpredictorsused intheestimationdropsbyroughly50%. Some readers may be concerned that our small shrinkage is reliant on the estimatedfattailintruereturns,andthatfattailedmodelsaretrickytoestimate. The“normaltruereturns”columnshouldalleviatethisconcern. Itassumesthat the degrees of freedom parameter is 100 and omits it from the estimation. The resulting shrinkage of 8.3% is somewhat smaller than the baseline, but leads to thesameconclusion: publicationbiasismodest. 4.3. EstimationonHand-CollectedData The final column (“hand-collected data”) of Table 4 examines our handcollecteddataset(Section3.4). Thisalternativedatashouldalleviatetheconcern thatourreplicationsdonotfullycapturethepublicationbiasintheoriginalpapers. The hand-collected data column shows that similar bias adjustments result fromusingtheoriginalreportedstatistics.Themeanshrinkageis9.9%,economicallyandstatisticallyclosetothebaselineestimateof12.5%. Indeed,theparametersareoverallsimilartothebaseline.Thehand-collected datashowsaweakerfattailwithν µ =6.56comparedtothebaselineestimateof 3.89, however both estimates are within one standard error of each other. This weaker fat tail comes with a larger σ µ, so the overall dispersion is similar: the hand-collecteddataleadstoacross-predictorstandarddeviationof0.70,notfar fromthebaselineestimateof0.66. The t-stat threshold t is estimated to be 1.88, just a touch lower than the cut 1.96thresholdusedina2-sided5%significancetest.Thislowerthresholdcomes fromthefactthat4ofourhand-collectedportfolioshavet-stats<1.96. These4 predictors come with additional tests, however, that produce statistical significance. For example, while Ikenberry, Lakonishok, and Vermaelen (1995) report that their repurchase announcement portfolios produce a modest t-stat of 1.5, theireventstudiesshowt-statsinexcessof6. The 77 hand-collected portfolios include both quintile and decile sorts, as wellasportfolioreturnsthatareadjustedforfactorexposureandcharacteristics. 27

Estimationsonthe47portfoliosthatdonotmakeadjustmentsledtosimilarresults. 4.4. Heterogeneityinbiasadjustments Themeanshrinkageisjust12%,butthecross-predictorstandarddeviationis somewhatlargeat10percentagepoints(Table4). Figure5shows,however,that modestshrinkageisagooddescriptionoftheestimatesoverall. Figure5showsahistogramoftheshrinkagedistribution,aswellastheidentitiesofthepredictors.Eachnamerepresentsonepredictor,andthuseachstackof namesrepresentsthenumberofpredictorswithinaparticularbinofshrinkage. The shrinkage distribution is right-skewed, with a large mass at low shrinkage. Thus, 78% of predictors have shrinkage less than 20%. Moreover, even the highshrinkagepredictorshaveonlyamoderateamountofpublicationbias. The maximumshrinkageamong156predictorsis38.6%. Figure 5 also illustrates the determinants of the predictor-level shrinkage. Portfolios with high return volatility (red or purple text) dominate the distribution above 20% shrinkage. This result is intuitive: portfolios with a lot of noise aremorelikelytohavehadluckyin-samplereturns,andthusexhibitmorepublicationbias(onaverage). Theoretically,thesamplelengthshouldalsoplayakeyroleintheamountof noise, and thus the magnitude of shrinkage (Equation (22)). However, we find that the empirical correlation between the sample lengths and return standard errorsisonlymildlynegative,at-0.14. It’sworthnotingthathighershrinkagedoesnotimplypoorbias-adjustedreturns. Highershrinkageportfolioshavelargerstandarderrors,andtheseportfoliosneedtohavehigherin-samplereturnsinordertomeetthepublicationt-stat threshold.Thehigherin-samplereturns,then,compensatesforthelargeshrinkage. Indeed,thehighestshrinkageportfolio,LaPorta(1996)’slongtermearnings growthpredictor(EPSForeLT),earnsarespectablebias-adjustedreturnof0.47% permonth. 28

Figure5:Distributionofpublicationbiasadjustments. Weestimateamodelof biasedpublication(Table1)on156long-shortportfolios(Table4). Shrinkageis definedby Bias-adjustedreturn=(1−[Shrinkage])×[In-SampleReturn] where the bias-adjusted return is calculated using Equations (19)-(20). Each namerepresentsoneportfolio. ThefullreferencesareinTableA.1. Publication biasisheterogeneousandrightskewed,butmodestoverall. 50 45 40 35 30 25 20 15 10 5 0 0 to 5 5 to 10 10 to 15 15 to 10 20 to 25 25 to 30 30 to 35 35 to 40 Shrinkage (%) tnuoC top quartile return volatility bottom quartile sample length AbnAccr Accruals both of the above BMent BPEBM ChATurn ChCOA ChCol ChFAccrua ChFinLiab ChInvento ChInvestI ChLTI ChNAnalys ChNCOA AdExp ChNWC AssetGrow ChPM BetaTailR ChTax CBOperPro CompDebtI ChBE ConvDebt ChBEtoA AssetTurn DebtIssua ChDeprToP BM DelBreadt ChRecomm CFPcash DivOmit DivInd CFPinc DownForec EPSRevisi DivInit EarnCons EarnSupBi DivYield EarnSurp GrGMToGrS EP EntMult GrossProf EPforecas ExclExp Herf ExchSwitc GrAdExp IndMom FailurePr GrCAPX Mscore GIndex GrEmp OperLever GrSaleToG GrLTNOA OrderBack Illiquidi GrSaleToG PctAcc IntanCFP Cash IndRetBig PctTotAcc IntanEP ConsRecom InvToRev RD InterMom EffFronti AnnounceR KZ RIO_IdioR Leverage FirmAgeMo DolVol Mom1m RetConglo Mom12m ForecastD IntanSP NOA RevenueSu Mom1813 IO_ShortI MaxRet AccrualsB NetDebtFi RoE Mom6m IntanBM Mom36m ChDRC NumEarnIn SalesGr NetEquity NetDebtPr Mom6mJunk ChOptVol PensionFu ShareIs1 OScore OperProf MomRev High52 PriceDela ShareIs5 PayYield Pscore MomVol IdioRisk BidAskSpr RealEstat ShortInte RDIPO SEO OrgCap ProfitMar CredRatDG Beta RoA SinStock ShareVol SmileSlop RDirtSurp RIO_BM IndIPO BetaSquar Seasonali Skew1 SurpriseR Spinoff SalesToPr RIO_Disp OptVol EPSForeLT ShareRepu VolSD Tax VolumeTre StdTurnov RIO_Turno VarCF NetPayout UpForecas XFIN ZScore ZeroTrade Tangibili Size VolMkt Price 29

5. Why is Publication Bias So Small? Our results may be surprising, especially to those who work in the crosssectional returns literature. Some might feel certain that, we must be mining thedata. Atleast,asacollectivewemustbe. But there are controls on the publication process that are designed to limit the negative effects of data-mining. And a priori, it’s hard to know which force dominates. Our estimator takes an empirical approach, and lets the data speak about whichforceisstronger. TheestimatorbelongstotheempiricalBayesfamily,and as such, it learns about a given predictor by studying the larger family of predictors(Efron(2012)). Thisfamilydisplaysconsiderabledispersion,muchmore dispersionthanwouldbeimpliedbypurenoise. Usingthisinformation,theestimatorconcludesthatthereisalotofsignalineachpredictor. This section explains the intuition behind our estimated bias adjustments. Section 5.1 shows that the mean bias adjustment is determined by the dispersionoftruereturns. Section5.2showshowthedispersionoftruereturnsisdeterminedbythedispersionofin-samplereturns. Section5.3finishesupbycomparingourbiasadjustedreturnswithMcLeanandPontiff(2016)’slowerbound. 5.1. Mean Shrinkage is Determined by the Dispersion of True Returns Our bias adjusted return comes from a complicated expression (Equations (19) and (20)), but plotting the bias adjustments reveals some intuition for how theadjustmentworks. Figure6plotsthebiasadjustmentsagainstthestandarderrorsoftheportfolio’sin-samplereturn. Thescattershowsaclearpattern: thelargerthestandard error,themoreshrinkageisrecommended. Thisresultisintuitive: morevolatile portfoliosorpublicationswithshortersamplesaremorelikelytohaveluckyinsamplereturns. Thus,theseluckyportfoliosrequirealargeradjustment. Indeed, the relationship between the standard error and shrinkage can be expressed in closed form for the normal approximation of our model. In this approximation, the shrinkage is a sort of noise-to-signal ratio, where the noise istheportfolio-specificstandarderror(Section3.2). Thenormalapproximation 30

Figure 6: Determinants of the bias adjustments. Each marker represents one portfoliofromourdatabaseof156predictors. Shrinkageisdefinedby Bias-adjustedreturn=(1−[Shrinkage])×[In-SampleReturn] wherethebias-adjustedreturnisEqns. (19)-(20). Thenormalapproximationis σ2 [Shrinkage] = i . i σˆ2 µ +σ2 i whereσ isthestandarderrorandσˆ2 istheestimateddispersionoftruereturns. i µ Thenormalapproximationworkswellformostportfolios.Theprimarydeterminantofthemeanshrinkageisσˆ2. µ 50 45 40 35 30 25 20 15 10 5 0 0 0.1 0.2 0.3 0.4 0.5 Standard Error )%( egaknirhS Bottom Two Terciles of In-Sample Return Top Tercile In-Sample Return normal approx mean shrinkage (solid line) describes the full shrinkage formula well for most of the portfolios, thoughitmissestheportfolioswithveryhighin-samplereturns(triangles). This deviation occurs because the full model has a fat tail in true returns, and these highreturnportfoliosaremorelikelytobelonginthetail. But overall, the normal approximation does a good job of capturing shrinkage. Indeed, our headline 12% shrinkage can be derived using this approximation. Plugginginthemeanstandarderrorof0.19%andourestimatedσˆµ =0.45, thetypicalshrinkageisapproximately σ2 0.192 i = =15% (24) σˆ2 µ +σ2 0.452+0.192 i Thisanalysisbegsthequestion: wheredoesourestimateofσˆµ =0.42come 31

from? 5.2. The Estimated Dispersion of True Returns is Determined bytheDispersionofIn-SampleReturns. We’ve seen that the mean shrinkage is determined by the estimated dispersionoftruereturnsσˆµ. Here, weshowthatσˆµ isidentifiedbythedispersionof in-samplereturns. This identification is illustrated in Figure 7, which plots the distribution of in-samplereturnsinthedata(bars)andmodel-implieddistributions(squares). The left panel shows our estimated model. The other panels show models that displaylargepublicationbias. Figure 7: Identification of the Dispersion of True Returns σ µ. Each panel illustratesthefitofadifferentmodel. Theleftpanelcomparesthedistributionof publishedreturnsinthedata(bars)withtheestimatedmodel(squares,Table4). Thedistributionoftruereturnsimpliedbythemodelisplottedforcomparison (dash dotted line). “Large bias 1” uses σ µ =0.075, but all other parameters remainthesame. “Largebias2”hasσ µ =0.075,t slope =3,andt cut =3.0. Bothlarge biasmodelsarepoorfitsforrighttailofthedistribution. 0.6 0.5 0.4 0.3 0.2 0.1 0 0 1 2 3 Return ycneuqerF Estimated 0.6 Published In-Sample Model: In-Sample 0.5 Model: True Mean Shrinkage = 12% 0.4 Log Like = -145.46 0.3 0.2 0.1 0 0 1 2 3 Return ycneuqerF Large Bias 1 0.6 0.5 Mean Shrinkage = 50% 0.4 Log Like = -356.37 0.3 0.2 0.1 0 0 1 2 3 Return ycneuqerF Large Bias 2 Mean Shrinkage = 50% Log Like = -244.68 The estimated model (left panel) is a tight fit for the published data. The modelhistogramcountsareclosetothedatathroughoutthedistribution. This tightfitcomesdespitethefactthatthemodelhasonlysixestimatedparameters, andthatthemodelalsomustfitthedistributionofstandarderrors(notshown). The left panel also shows the distribution of true returns implied by the model (dash-dotline). Truereturnsarequiteclosetothein-samplereturns,leadingto 32

thesmallmeanshrinkageofjust12%. The middle panel plots the distribution of in-sample returns implied by a model with large bias. This model deviates from the estimated model only in that σ µ =0.075, compared to the estimated σ µ =0.421. σ µ =0.075 is chosen in ordertoachieveameanshrinkageof0.50. Thisshrinkageisimportantbecause McLeanandPontiff(2016)findthatpost-publicationreturnsarelowerthaninsample returns by 58%. Thus, σ µ = 0.075 is required to assign the bulk of this declinetopublicationbias. The middle panel shows that this large bias model is a poor fit for the data. This model fails to capture the dispersion of in-sample returns. Our estimator sees much more than just the dispersion however. As we use maximum likelihood,theestimatorseesthefitofeveryin-samplereturnbin,andtheexcessively high counts for the low return bins as well as the excessively low counts for the high returns bins are all penalized by the estimator. Indeed, the log-likelihood of this model is more than 200 log points lower than our maximum likelihood estimate. One might argue that the large bias model needs to have other parameters adjusted to fit the data. One adjustment consistent with the idea that the data exhibitsalargebiasisalargeincreaseint andalargedecreaseint —that cut slope is,thejournalsexhibitastrongpreferenceforlarget-stats. The right panel of Figure 7 shows that increasing t and decreasing t cut slope does not fit the data either. The panel assumes a high t threshold midpoint of t cut =3.00 and a relatively shallow slope of t slope =3, in addition to σ µ =0.075. This strong preference for large t-stats improves the fit on the left side of the distribution, but overall the fit is still poor, with the log-likelihood is still about 100logpointsbelowourmaximumlikelihoodestimate. Experimentswithother strongpreferencesforhight-statsledtosimilarresults. This identification discussion begs the question: does the estimated model fittheotherdimensionsofthedata? Figure8showsthattheanswerisyes. Thefigure’s4panelsplotthedistributionoft-stats, in-samplereturns, standard errors, as well as a plot that illustrates the correlation between in-sample returnsandstandarderrors. All4panelsofshowthattheestimatedmodelcapturesthedataverywell. 33

Figure 8: Model fit. We simulate the model using estimated parameter values (Table 4) and compare the distribution of observables with those from our database of 156 predictors (Table 2). The t-stat thresh uses estimated parameter values. The model fits all observable distributions very well, including the correlationbetweenin-samplereturnsandstandarderror(bottomright). 0.2 0.15 0.1 0.05 0 0 2 4 6 8 10 12 14 t-stat ycneuqerF 1 0.8 0.6 0.4 0.2 0 buP fo ytilibaborP 0.3 Published data Estimation-implied 0.25 t-stat thresh (right) 0.2 0.15 0.1 0.05 0 -1 0 1 2 3 4 In-Sample Return ycneuqerF Published Data Estimation-Implied 0.2 0.15 0.1 0.05 0 0 0.1 0.2 0.3 0.4 0.5 0.6 SE ycneuqerF 4 Published Data Estimation-Implied 3 2 1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 Standard Error nruteR elpmaS-nI Published Data Estimation-Implied t-stat thresh mid 5.3. External Verification: McLean and Pontiff (2016)’s Lower Bound We’ve shown that the model fits data that it algorithmically targets. But is thereawaytobringtobeardatafromoutsidetheestimation? The natural external test would be to compare our estimated shrinkage to out-of-sample returns. Out-of-sample returns, however, are polluted by tradingeffects,sinceinvestorsthatreadthepublicationsmaydecidetochangetheir portfolioallocations. McLean and Pontiff (2016) (MP) develop a clever way to isolate trading effects. Assumingthatpapersarelesswidelyknownbeforepublication,thereturn betweentheendoftheoriginalsampleandthepublicationdateshouldexhibita limitedamountoftradingeffects. Thus,themeanreturninthis“between”sampleservesasalowerboundonthepublicationbiasadjustedreturn. Figure 9 examines MP’s lower bound. The figure shows scatterplots of bias- 34

adjusted returns against the in-sample return, as well as the mean returns betweentheendoftheoriginalsampleandpublication.Forcomparison,thefigure alsoplotsthenaiveprediction: in-samplereturn=truereturn. Figure 9: Bias adjusted returns and returns after the original in-sample period. Each marker represents one long-short portfolio. The naive prediction (solid line) assumes that the true return is equal to the in-sample return. “Returnbetweensampleandpub”(circles)arethemeanreturnsbetweentheendof theoriginalpaper’ssampleandthepublicationdate. Thebias-adjustedprediction(triangles)useEquations(19)-(20)andestimatedparameters(Table4). Low standard error portfolios are those with standard error below the median. The bias-adjustedreturnsareconsistentwiththelowerboundimpliedbythemean returnbetweensampleandpublication. 4 3 2 1 0 -1 0 0.5 1 1.5 2 2.5 In-Sample Return nruteR erutuF / detciderP Low Standard Error 4 NaivePrediction Bias-AdjustedReturn 3 ReturnBetweenSampleandPub 2 1 0 -1 -2 -3 0 1 2 3 In-Sample Return nruteR erutuF / detciderP High Standard Error The plot shows that bias-adjusted returns are very similar to naive predictions. This, essentially, is the main message of our paper: publication bias is modest. Thismodestbiasisparticularlyevidentinlow-standarderrorportfolios (leftpanel). More importantly, the figure provides external validation of our bias adjustment. Thecirclesrepresentthereturnsbetweentheendofthesampleandpublication. High publication bias implies that these circles would be symmetricallyspreadacross0. Instead, thecirclesaremoreorlesssymmetricallyspread aroundthenaivepredictionline. Moreover,ourbiasadjustedreturnsareslightlyabovethemiddleofthecloud ofcircles. Averagingacrossthebluecircleswefindthatourmeanbias-adjusted return is consistent with MP’s lower bound. The average return in the between 35

sample is 0.56% per month, slightly below our mean bias-adjusted return of 0.63%. 6. Implications for the Anomaly Zoo Theassetpricingliteraturehasuncoveredhundredsofpatternsinthecrosssectionofstockreturns. Recentresearchhasaimedtoplacesomeorderonthis zoo of anomalies (Cochrane (2011), Harvey, Liu, and Zhu (2015), Kozak, Nagel, andSantosh(2017),Feng,Giglio,andXiu(2017)). Ourbiasadjustedreturnsimplythat(1)correctingfordataminingdoesnot reducethesizeoftheanomalyzoo,and(2)muchofthepredictabilityatthetime ofpublicationwasduetomispricing. Sections6.1and6.2discusstheseimplications,respectively. 6.1. HypothesisTestsAdjustedforPublicationBias Thesmallbiasadjustmentssuggestthatthezooofanomaliescannotbesimply attributed to publication bias. Here, we look more closely at the question andshowthatnearly100%ofpublishedanomaliesaretrueusingmultipletestingstatistics. To demonstrate this, we use our estimated model to calculate the false discoveryrate(FDR),oneofthemultipletestingstatisticsrecommendedbyHarvey, Liu,andZhu(2015)(HLZ).WefocusontheFDRinsteadofthefamilywiseerror ratebecauseofitssimpleinterpretation: theFDRistheshareofanomaliesthat arestatisticalfigments. Wedefineanullpredictorasonewithnon-positivetruereturns(µ ≤0). This i definition stays close to the classical definition and also to HLZ, both of which definethenullasµ =0. Thisnullisalsoimportantasitisusedinpopularmuli tipletestingadjustments(forexample,BonferroniandBenjaminiandHochberg (1995)). In contrast to these approaches, our model considers portfolios with worse-than-zero true returns, and thus we must label µ <0 as null in addition i toµ =0.9 i 9Analternativetousingµ ≤0asthenullistoestimateamodelinwhichµ isdrawnfrom i i astrictlypositivedistributionandapointmassatzero. Thepointmassatzeroservesasimilar functionasthedistributionofnegativeµ inourmodel. i 36

Usingthisdefinitionofanullpredictor,wecancalculatefalsediscoveryrates bysimulatingtheestimatedmodel. Figure10illustratesthiscalculation. Thetop panel shows the distribution of narrative portfolios’ true returns againstt-stats. Nullportfolios, thatis, portfolioswithnon-positivetruereturnsarehighlighted inred. Figure10:MultipleTestsoftheNullofNon-PositiveTrueReturns. Wesimulate narrativeportfoliosusingourestimatedmodel(Table4). Thetoppanelshowsa scatterplotof10,000truereturnsagainstt-stats. Thefalsediscoveryrate(FDR) for a given t hurdle is the fraction of predictors exceeding the hurdle that have non-positive true returns (red dots). Incorporating information from multiple testsleadstothet-hurdlesgivenbythegreenlines,whicharemorelenientt-stat hurdlethanthetraditional1.96(greydashedline). 2 1.5 1 0.5 0 -0.5 -1 -1.5 0 1 2 3 4 5 6 t-stat nruteR eurT Null (false discovery) t hurdle: FDR = 5% t hurdle: FDR = 1% 15 10 5 0 0 1 2 3 4 5 6 t-stat Hurdle )%( etaR yrevocsiD eslaF t hurdle: FDR = 5% t hurdle: FDR = 1% Forportfolioswitht-statsof0.5,theprobabilityofbeingnullisabout50%,as indicated by the even split between red and light blue dots near the left side of thepanel. Thecloudofdots,however,isupwardsloping,andthus,highert-stat portfoliosaremorelikelytobenon-null. Thispatternismorepreciselydescribedinthebottompanel.Thepanelplots 37

the FDR as a function of the t-stat hurdle. Even the extremely generous hurdle of0leadstoalowFDRof12.7%. Increasingthet-stathurdledecreasesthefalse discoveriessharply.Atat-stathurdleof0.85wealreadyhaveanFDRof5%,oneof theFDRvaluesinrecommendedbyHLZ.Raisingthet-stathurdleto1.79reduces theFDRto1%,HLZ’salternativerecommendation. Thus,ourresultssuggestthatthetraditionalt-stathurdleof1.96couldactuallybeloosened.Evenat-stathurdleof0.85effectivelycontrolsforfalsediscoveries,giventhattheportfoliohasatop-tierqualitynarrative. Thissurprisingresult comesfromthefactthatweestimatethedispersionofnarrativetruereturnsto beverylarge. Thislargedispersionimpliesthatthet-statisastrongsignalabout theunderlyingtruereturn,thecloudofdotsinthetoppanelofFigure10isupwardsloping,andthusalarget-statisnotrequiredforconcludingthetruereturn ispositive. Incontrast,singlehypothesistestsdonotallowforanyinferencesaboutthe dispersionoftruereturns. Withasinglepredictor,theonlyreasonableapproach is to assume that the predictor is useless, leading to a high t-stat hurdle. Less structuredmultiple-testingadjustmentssuchastheBonferroniandBenjamini- Hochbergadjustmentsalsodonotestimatethedistributionoftruereturnsand insteadassumetheworstcase,asweexplaininAppendixA.4. An important caveat is that our results apply only to predictors that are judgedtohavequalitynarratives,thatis,softcharacteristicsthatsatisfythejournal review process. Thus, ourresults donot imply thata randomly data-mined portfoliowithat-statof0.85is95%likelytohavepositivetruereturns, andour low FDR estimates are consistent with Chordia, Goyal, and Saretto (2017)’s results regarding randomly generated signals. Similarly, our results do not imply thatjournalsshouldconsiderlooseningtheirt-statrestrictionswithoutcarefully maintainingtheirnarrativecontrols. Nevertheless, our results do apply to predictors that are published in peerreviewed journals. Indeed, as peer-reviewed journals only allow narratives that meetathresholdcenteredaround1.5(Table4), almostallpublishedpredictors are true. According to our estimates, the FDR among published predictors is a tiny 1.5%. This FDR would increase if we regard the three predictors that we omittedduetoreplicationfailuresasfalsediscoveries. However,evenassuming thesearefalsediscoveriesleadstoalowFDRof4%. One interpretation of the low estimated t-stat hurdles is that the traditional 38

null hypothesis of µ =0 is inadequate. This null describes only a tiny portion i ofnarrativepredictors. Asaresult,non-nullpredictorsarenotunusual,andthe null does not help separate interesting cases from typical ones. In this setting, one may want to use an “empirical null” that is designed to generate unusual cases(Efron(2012). WediscussonesuchempiricalnullinAppendixA.5. Our results contrast with HLZ, who find that t-stat hurdles close to 3.0 are requiredtoreducetheFDRbelow1%. HLZ’sdataissubstantiallydifferentthan ours, however. While our dataset includes only predictors that demonstrate return predictability, HLZ’s dataset is comprised of asset pricing factors, broadly defined.Perhapsasaresult,thedispersionoft-statisticsislargerinourdata.The 90thpercentilet-statisticinoursampleis8.0,comparedtothe90thpercentileof 6.3inHLZ. Thereareotherdifferencesinmethodologywhichmaycontributetothedeviation in results, however. Our model uses both mean returns and standard errors, while HLZ consider only the t-stat. HLZ’s model assumes a mixture distribution for true returns, while ours assumes a single fat-tailed distribution. A clearreconciliationofourlowt-stathurdleandHLZ’st-stathurdlesabove3.0is, inourview,animportantquestionforfutureresearch. 6.2. ImpliedMispricing Our estimation results thus far are negative. We find that publication bias cannotaccountforthezooofpublishedstockreturnanomalies. In this section, we present evidence in favor of a more positive conclusion. CombiningourestimationwiththeempiricalmethodologyofMcLeanandPontiff(2016)andMarquering,Nisser,andValla(2006),wefindevidencesuggesting thatmispricingplaysanimportantrolethroughouttheanomalyzoo. McLean and Pontiff (2016) (MP) and Marquering, Nisser, and Valla (2006) buildontheinsightthattheaveragereturnpostpublicationisinformativeabout the nature of the cross-sectional predictability. If predictability is due to mispricing, postpublicationreturnsshouldbepoor, astradersbidupunderpriced assetsandavoidoverpricedones. Similarly,ifpredictabilityisduetopublication bias,post-publicationreturnsshouldbepoorasthepre-publicationpredictabilitywasastatisticalfigment. Ontheotherhand,risk-basedstoriesdonotprovide aclearprediction. 39

This logic leads to the decompositions of in-sample returns seen in Figure 11. The figure decomposes the average in-sample return across predictors into (1)publicationbias(2)non-statisticaldeterioration,and(3)thepost-publication return. Figure 11: Implied mispricing. This chart decomposes the average in-sample returnacrosspredictorsintopublicationbias,non-statisticaldeterioration,and thepost-publicationreturn.Publicationbiasistheaveragein-samplereturnminustheaveragebias-adjustedreturn(Equations(19)-(20)). Post-publicationreturnisthemeanreturninthesampleafterpublication. Non-statisticaldeteriorationisthedifferencebetweentheaveragebiasadjustedreturnandtheaverage post-publicationreturn. Eachbarcomputesaverageswithinasubsetofthepredictors. “Low S.E.” consists of portfolios with below the median standard error, andsimilarlyfor“lowt-stat.” Asignificantportionofin-samplereturnsisdueto non-statisticaldeterioration,suggestingthatmispricingisimportant. 1.2 1 0.8 0.6 0.4 0.2 0 Low S.E. High S.E. Low t-stat High t-stat nruteR elpmaS-nI Publication bias Non-statistical deterioration Post-publication return Thedecompositioncomesfromcomputingaveragereturnsofdifferenttypes andtakingdifferences.Publicationbiasisthedifferencebetweentheaverageinsample return and the average bias adjusted return (Equations (19) and (20)). Post-publicationreturnsaretheaveragereturnsafterthepublicationdate. The non-statisticaldeteriorationisthedifferencebetweentheaveragebias-adjusted returnandtheaveragepost-publicationreturn. 40

Thefigureshowsthatasignificantportionofin-samplereturnsisduetononstatistical deterioration. The average post-publication return is 25 basis points permonthlowerthantheaveragebias-adjustedreturn. Thisnon-statisticaldeteriorationaccountsforasignificant34%oftheaveragein-samplereturn. Nonstatisticaldeteriorationislargestforhight-stat,lowstandarderror,andhighinsamplereturnportfolios,consistentwithMPandthehypothesisthatmispricing istheunderlyingdriverofpredictability. OurresultsgobeyondMP,however,inseveralways.WhileMPplaceanupper boundonpublicationbias,andthusalowerboundonnon-statisticaldeterioration, our bias adjustments provide direct estimates of both. MP’s upper bound alsorunsintoacoupletheoreticalconcerns: namelyitassumesthatthereisno selection happening between the end of the in-sample period and publication. Our estimator avoids these concerns by explicitly modeling and estimating the selection process. Finally, our dataset is nearly twice the size of MP’s, which is important considering how volatile stock returns are and how short the postpublicationperiodcanbe. Theserefinementsmeanthatwecanmakeinferencesonsubsamplesofpredictors with confidence. This increased precision is highlighted in Figure 12, whichplotsthebootstrappeddistributionofnon-statisticaldeterioration. Thefiguresplitsthedataintoportfolioswithstandarderrorsbelow themedian(leftpanel)andthoseabove(rightpanel). Inbothpanels,the5thpercentile ofthebootstrappedstandarderrorsareveryfarfromzero. Thuswecanbeconfidentthatasignificantshareofin-samplereturnsisduetonon-statisticaldeterioration. Indeed,thehypothesisthatpublicationbiascanaccountforallofthe deteriorationissoundlyrejectedforbothlowandhighstandarderrorportfolios (p-values<0.0001). 41

Figure 12: Implied mispricing: bootstrapped distribution. We resample the data10,000timesandrunourestimatoroneachresampling. LowS.E.portfolios have standard errors below the median. Mean non-statistical deterioration is theaveragebias-adjustedin-samplereturnminustheaveragepost-publication return, all divided by the average in-sample return. The hypothesis that publication bias accounts for all deterioration in returns post-publication is soundly rejected,suggestingthatmispricingisimportant. 0.12 0.1 0.08 0.06 0.04 0.02 0 -20 0 20 40 60 80 100 Mean Non-Statistical Deterioration (as a Percent of the Mean In-Sample Return) ycneuqerF Low S.E. Portfolios 0.1 0.08 0.06 0.04 5th pct 0.02 0 -20 0 20 40 60 80 100 Mean Non-Statistical Deterioration (as a Percent of the Mean In-Sample Return) ycneuqerF High S.E. Portfolios 5th pct 42

7. Conclusion We find that the net effect of publication bias on cross-sectional stock predictorsismodest. Theseresultssuggestthateditorsandrefereesprovideanimportantcontrolonourcollectiveminingofthedata, leadingtothediscoveryof a multitude of portfolios with high returns and low market risk. These high returns,however,areshort-lived,astradersquicklyactonthepublicationofreturn predictabilityandeliminatemispricing. Ourresults,combinedwithacoupleotherrecentpapers,provideacomplete accountingforthereturnsoftheanomalyzoo. Wefindthatthetypicalanomaly return of 8% per year is 12% publication bias. McLean and Pontiff (2016) show thatanother35%ismispricingthatcanbetradedaway. ChenandVelikov(2017) completethestory,showingthatmuchoftheremaining53%canbeaccounted forbytradingcosts. 43

A. Appendix A.1. AdditionalDetailsontheReplicatedData Table A.1: Description of Anomaly Construction. This table provides details of the construction of return predictors used in the paper. Data come from the CRSP stock return database, Compustat North America Annual and Quarterly databases, IBES earnings estimates database, OptionMetrics,ThomsonSDCandanumberofadditionaldatabasesnotedinthedescriptionsofspecificanomalies. Ourfinaldatabaseissetupat monthlyfrequency.WelagannualCompustatdatabyfivemonthsandquarterlyCompustatdataby3monthstoassureavailabilityofrelevantdata atthetimeoftrading. Acronym Description Author(s) Pubyear SampleStart SampleEnd Description AbnAccr AbnormalAccruals Xie 2001 1971 1992 DefineAccrualsasnetincome(ib)minusoperatingcashflow(oancf), dividedbyaveragetotalassets(at)foryearst-1andt.Ifoancfismissing,replaceoperatingcashflowwithfundsfromoperations(fopt)minustheannualchangeintotalcurrentassets(act)plustheannual changeincashandshort-terminvestments(che)plustheannual changeincurrentliabilities(lct)minustheannualchangeindebtin currentliabilities(dlc). Foreachyeart,regressAccrualson: theinverseofaveragetotalassetsforyearst-1andt,thechangeinrevenue (sale)fromyeart-1totdividedbyaveragetotalassets,properyplant andequipment(ppegt)dividedbyaveragetotalassets,industrydummiesforFama-French’s48industryclassification. AbnormalAccrual istheresidualfromthiscross-sectionalregression. Accruals Accruals Sloan 1996 1962 1991 Annualchangeincurrenttotalassets(act)minusannualchangein cashandshort-terminvestements(che)minusannualchangeincurrentliabilities(lct)minusannualchangeindebtincurrentliabilities (dlc)minuschangeinincometaxes(txp).Alldividedbyaveragetotal assets(at)overthisyearandlastyear.Excludeifabs(prc)<5. AccrualsBM Book-to-marketandaccruals BartovandKim 2004 1980 1998 Binaryvariableequalto1ifstockisinthehighestAccrualquintileand thelowestBMquintile,andequalto0ifstockisinthelowestAccrual quintileandthehighestBMquintile. Excludeifbookequity(ceq)is negative. AdExp AdvertisingExpense Chanetal 2001 1975 1996 Advertising expense (xad) over market value of equity (shrout*abs(prc)) AnnounceR Earningsannouncementreturn Chanetal 1996 1977 1992 GetannouncementdateforquarterlyearningsfromIBES(fpi=6). AnnouncementReturnisthesumof(ret-mktrf+rf)fromoneday beforeanearningsannouncementto2daysaftertheannouncement. AssetGrowth AssetGrowth Cooperetal 2008 1968 2003 Annualgrowthrateoftotalassets(at) Continuedonnextpage 44

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description AssetTurnover AssetTurnover Soliman 2008 1984 2002 Sales (sale) divided by two year average of net operating assets. Net operating assets is the sum of receivables (rect), inventories (invt),currentassetsother(aco),netproperty,plantsandequipment (ppent)andintangibles(intan),minusaccountspayable(ap),other currentliabilities(lco)andotherliabilities(lo).Excludeifabs(prc)< 5orAssetTurnover<0. Beta CAPMbeta FamaandMacBeth 1973 1926 1968 Coefficientofa60-monthrollingwindowregressionofmonthlystock returnsminustheriskfreerateonmarketreturnminustheriskfree rate(ewretd-rf).Excludeifestimatebasedonlessthan20monthsof returns. BetaSquared CAPMbetasqured FamaandMacBeth 1973 1926 1968 SquareofBeta(definedabove). BetaTailRisk Tailriskbeta KellyandJiang 2014 1963 2010 Eachmonth,computethe5thpercentileoverdailyreturnsoverall firms. Foralldailyreturnobservationswithreturnbelowthat5th percentile,computetheaverageof(log(ret/5thpercentileofcrosssectionalreturndistribution).CallthataveragetailEX.BetaTailRiskis thecoefficientofa120-monthrollingregressionofafirm’sstockreturnontailEX.Excludeifpricelessthan5orsharecodegreaterthan 11. BidAskSpread Bid-askspread AmihudandMendelsohn 1986 1961 1980 Spread estimates from Shane Corwin’s website (https://www3.nd.edu/scorwin/)dividedbyprice(abs(prc)). BM Booktomarket FamaandFrench 1992 1963 1990 Logofannualbookequity(ceq)overmarketequity(seeabove). BMent EnterprisecomponentofBM PenmanRichardson 2007 1961 2001 (ceq+che-dltt-dlc-dc-dvpa+tstkp)/(mve\_c+che-dltt-dlc- Tuna dc-dvpa+tstkp).Excludeifpricelessthan5. BPEBM LeveragecomponentofBM PenmanRichardson 2007 1961 2002 BP-EBM,whereBP=(ceq+tstkp-dvpa)/(shrout*abs(prc)), and Tuna EBMisdefinedabove.Excludeifpricelessthan5. Cash Cashtoassets Palazzo 2012 1972 2009 Ratioofquarterlycashandshort-terminvestments(cheq)andtotal assets(atq). CBOperProf Cash-basedoperatingprofitability Balletal 2016 1963 2014 Revenue(revt)minuscost(cogs)-(administrativeexpenses(xsga)- R&Dexpenses(xrd))minusannualchangeinreceivables(rect),annualchangeininvestment(invt)andannualchangeinprepaidexpenses,plusannualchangeincurrentdeferredrevenue(drc),longtermdeferredrevenue(drlt),accountspayable(ap)andaccruedexpenses(xacc),alldividedbytotalassets(at)inyeart-1. Replaceall variablesinthenumeratorwith0iftheyaremissing.Excludeifshare codeisgreater11,marketvalueofequity,BMortotalassetsaremissing,orifSICcodebetween6000and6999. Continuedonnextpage 45

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description CFPcash OperatingCashflowstoprice Desai,Rajgopal,and 2004 1973 1997 Operatingcash-flow(oancf)dividedbymarketvalueofequity.Ifop- Venkatachalam eratingcash-flowismissing,replacebydifferencebetweenetincome (ib)andlevelofaccruals,wherethelatteristheannualchangeincurrentassets(act)minustheannualchangeincashandshort-terminvestments(che),minustheannualchangeincurrentliabilities(lct) plustheannualchangeindebtincurrentliabilities(dlc)plustheannualchangeinpayableincometaxes(txp)plusdepreciation(dp). CFPinc Cashflowtomarket Lakonishoketal 1994 1968 1990 Netincome(ib)plusdepreciation(dp)dividedbymarketequity.ExcludeNASDAQstocks. ChATurn ChangeinAssetTurnover Soliman 2008 1984 2002 AnnualchangeinAssetTurnover(definedabove).Excludeifpriceless than5. ChCOA Changeincurrentoperating Richardsonetal 2005 1962 2001 Differenceincurrentoperatingassets(totalcurrentassets(act)miassets nuscashandshort-terminvestments(che))betweenyearst-1andt, scaledbyaveragetotalassets(at)inyearst-1andt. ChCol Changeincurrentoperating Richardsonetal 2005 1962 2001 Differenceincurrentoperatingliabilities(totalcurrentliabilities(lct) liabilities minusdebtincurrentliabilities(dlc))betweenyearst-1andt,scaled byaveragetotalassets(at)inyearst-1andt. ChDeprToPPE Changeindepreciationtogross HolthausenandLarcker 1992 1978 1988 Annualpercentagechangeintheratioofdepreciation(dp)toprop- PPE erty,plantandequipment(ppent). ChDRC DeferredRevenue PrakashandSinha 2012 2002 2007 Annualchangeindeferredrevenue(drc)scaledbyaveragetotalassets(at)int-1andt. Excludeifnegativebookequity(ceq),deferred revenueequalto0inbothyears,revenuelessthan5m,orSICcode between6000and6999. ChEQ SustainableGrowth LockwoodandPrombutr 2010 1964 2007 Ratioofbookequity(ceq)tobookequityinthepreviousyear.Include onlyifbookequityispositivethisyearandlastyear. ChEqu Changeinequity Richardsonetal 2005 1962 2001 Differenceinbookequity(ceq)betweenyearst-1andt,scaledbyaveragetotalassets(at)inyearst-1andt. ChFAccrual ChangeinForecastandAccrual BarthandHutton 2004 1981 1996 WithinupperhalfofAccrualsdistribution,equalto1ifmeanearnings estimateincreasedrelativetothepreviousmonth.0ifitdecreased. ChFinLiab Changeinfinancialliabilities Richardsonetal 2005 1962 2001 Differenceinfinancialliabilities(sumoflong-termdebt(dltt),currentliabilitites(dlc)andpreferredstock(pstk))betweenyearst-1and t,scaledbyaveragetotalassets(at)inyearst-1andt. ChInventory InventoryGrowth ThomasandZhang 2002 1970 1997 12monthchangeininventory(invt)dividedbyaveragetotalassets. ChInvestInd Changeincapitalinv(indadj) AbarbanellandBushee 1998 1974 1988 Growthincapitalexpenditure(capx)minusaveragegrowthincapitalexpenditureinthesameindustry(two-digitSIC).Ifcapxismissing,capitalexpenditureisdefinedastheannualchangeinproperty, plantandequipment(ppent).Capitalexpendituregrowthisdefined asthepercentagegrowthofcapxtodayrelativetotheaveragecapx overtheprevioustwoyears(.5*(capx$_t-1$+capx$_t-2$),oraspercentagegrowthrelativetothepreviousyearonlyift-2ismissing. Continuedonnextpage 46

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description ChLTI Changeinlong-terminvestment Richardsonetal 2005 1962 2001 Differenceininvestmentandadvances(ivao)betweenyearst-1and t,scaledbyaveragetotalassets(at)inyearst-1andt. ChNAnalyst DeclineinAnalystCoverage Scherbina 2008 1982 2005 Binaryvariableequalto1ifthenumberofanalysts(numest)fornext quarter’sEPSestimatedecreasedrelativetothreemonthsago,and0 ifitincreased. ChNCOA ChangeinNoncurrentOperating Soliman 2008 1984 2002 Twelve-monthchangeinnoncurrentoperatingassets. Noncurrent Assets operatingassetsis((at-act-ivao)-(lt-dlc-dltt))/at. ChNWC ChangeinNetWorkingCapital Soliman 2008 1984 2002 Twelve-monthchangeinnetworkingcapital.Networkingcapitalis( (act-che)-(lct-dlc))/at ChOptVol OptionVolumerelativetorecent JohnsonandSo 2012 1996 2010 BasedoffofOptionVolume1.OptionVolume2=OptionVolume1/avaverage erageofOptionVolume1frommonthst-6tot-1. ChPM ChangeinProfitMargin Soliman 2008 1984 2002 AnnualchangeinprofitmarginPM(profitmargindefinedbelow). Excludeifpricelessthan5. ChRecomm Changeinrecommendation JegadeeshKimKrische 2004 1985 1998 (AsinMP).Ifananalystissuesanewstrongbuyrecommendation Lee (ireccd==1),weassignavalueof1tothatevent,ifananalystissues anyotherchangeinrecommendation,weassignavalueof-1;weassign0iftherecommendationisunchanged.Thefinalvariableisthe averageovertheconstructedvariableoverallanalystseachmonth. ChTax ChangeinTaxes ThomasandZhang 2011 1977 2006 4-quarterchangeinquarterlytotaltaxes(txtq),scaledbylaggedtotal assets(at). CompDebtI Compositedebtissuance LyandresSunZhang 2008 1970 2005 Logoflong-termdebt(dltt)plusdebtincurrentliabilties(dlc)minus logofthesamevariable5yearsago. ConsRecomm ConsensusRecommendation BarberLehavy 2001 1985 1997 Binaryvariableifthemonthlymeanofrecommendations(ireccd) MicNicholsTrueman overanalystsisgreaterthan3,and0ifitislessorequalthan1.5. ConvDebt Convertibledebtindicator Valta 2016 1985 2012 Binaryvariableequalto1ifdeferredcharges(dc)greaterthan0or commonsharesreservedforconvertibledebt(cshrc)greaterthan0. CredRatDG CreditRatingDowngrade DichevandPiotroski 2001 1970 1997 Adowngradehappensifcreditrating(splticrm)decreasedbyatleast onenotchrelativetothepreviousmonth.CredRatDG=1ifadowngradehappenedoverthepast3months. DebtIssuance DebtIssuance SpiessandAffleck-Graves 1999 1975 1989 Equalto1ifdebtissuance(dltis)greater0and0otherwise.Excludeif sharecode>11ormissingbook-to-market. DelBreadth Breadthofownership Chen,HongandStein 2002 1979 1998 Quarterly change in the number of institutional owners (numinstowners)from13Fdata. Excludeifinthelowestquintileofstocks bymarketvalueofequity(basedonNYSEstocksonly). DivInd Dividends HartzmarkandSalomon 2013 1927 2011 Binaryvariableequalto1ifreturnwithdividends(ret)isgreaterthan returnwithoutdividends(retx)11monthsagoor2monthsago,and 0otherwiseorifpricelessthan5. Continuedonnextpage 47

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description DivInit DividendInitiation MichaelyThalerWomack 1995 1964 1988 Definedividendinitiationashavingpaidadividendinmontht(divamt>0)andnothavingpaidadividendinthe24precedingmonths. DivInitisequalto1ifadividendwasinitiatedinthepast12months and0otherwise.Excludeifsharecodegreater11anduseNYSEstocks only. DivOmit DividendOmission MichaelyThalerWomack 1995 1964 1988 Definedividendomissionasnothavingpaidadividendinthecurrent monthorthetwoprecedingmonths,buthavingpaiddividendsinthe 3,6,9,12,15,18monthsbefore. DivOmitisequalto1ifadividend wasomittedintheprevious12monthsand0otherwise. DivYield DividendYield Naranjoetal 1998 1963 1994 4timeslatestdividend(divamt)dividedbyprice(prc). Includeonly ifdividendhasbeenpaidinallofthepast4quarters. DolVol Pasttradingvolume BrennanChordia 1998 1966 1995 Log of two-month lagged trading volume (vol) times two-month Subrahmanyam laggedprice(prc). DownForecast DownforecastEPS BarberLehavy 2001 1985 1997 Binaryvariableequalto1ifmeanearningsforecast(meanest)de- MicNicholsTrueman creasedoverthepastmonth. EarnCons EarningsConsistency Alwathainani 2009 1971 2002 Averageearningsgrowthoverprevious48months.Earningsgrowthis definedasEPS(epspx)minusEPS12monthsagodividedbyaverage EPS12and24monthsago.Excludeifpricelessthan5,absolutevalue of12monthearningsgrowthgreater600%,orearningsgrowthand earningsgrowth12monthsagohavedifferentsigns. EarnSupBig Earningssurpriseofbigfirms Hou 2007 1972 2001 AveragemonthlyvalueofEarningsSurprise(definedabove)ofthe 30%largestcompaniesbymarketvalueofequityinthesameFama- French48industry. Excludethelargest30%ofcompaniesforEarn- SupBig(nottocomputetheanomaly) EarnSurp EarningsSurprise Fosteretal 1984 1974 1981 EPS(epspxq)minusEPStwelvemonthsago-Drift,scaledbystandarddeviationofthatexpression.Driftistheaverageearningsgrowth (EPS-EPStwelvemonthsago)overthepasttwoyears. Excludeif pricelessthan5 EffFrontier Efficientfrontierindex NguyenandSwanson 2009 1980 2003 Frontieristheresidualofaregressionoflog(BM)onlog(bookequity(ceq)),long-termdebt(dltt)toassets(at),capitalexpenditures (capx)torevenue(sale),R&Dexpense(xrd)torevenue,advertising expense(xad)torevenue,propertyplantandequipment(ppent)to assets,EBIT(ebitda)toassets,anddummiesforFama-French’s48industrydefinitions. Regressionisupdatedeachmonthwitharolling windowof60months. EntMult EnterpriseMultiple LoughranandWellman 2011 1963 2009 Marketvalueofequity+long-termdebt(dltt)+debtincurrentliabilities(dlc)+deferredcharges(dc)-cashandshort-terminvestments (che),dividedbyoperatingincome(oibdp).Excludeifmissingbook equityornegativeoperatingincome. EP Earnings-to-PriceRatio Basu 1977 1957 1971 ib/lag(marketvalueofequity,6months).NYSEstocksonly.Exclude ifEP<0. LagsimulatestheDec31marketequityusedinoriginal paper Continuedonnextpage 48

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description EPforecast EarningsForecast Elgers,LoandPfeiffer 2001 1982 1998 Meanearningsestimate(meanest)fornextquarter’searningsdivided bystockprice(prc).Excludeifpricelessthan1. EPSForeLT Long-termEPSforecast LaPorta 1996 1983 1990 Long-termearningsforecast(fgr5yr)laggedbytwelvemonths. Excludeifbookequity(ceq),netincome(ib),deferredtaxes(txdi),dividends(dvp),revenue(sale)ordepreciation(dp)ismissing. EPSRevisions Earningsforecastrevisions Chanetal 1996 1977 1992 Definerevisionsasthechangeinthemeanearningsestimate(meanest)forthenextquarterfrommontht-1tot,scaledbystockpricein montht-1.REV6isthesumofthatvariablefrommonthst-6tot. ExchSwitch ExchangeSwitch DharanandIkenberry 1995 1962 1990 Binaryvariableequalto1ifafirmswitchedfromAMEXorNASDAQ toNYSEwithinthepastyear,orfromNASDAQtoAMEXwithinthe pastyear. ExclExp ExcludedExpenses Doyle,Lundholmand 2003 1988 1999 Difference between unadjusted earnings (EPSActualUnadj) from Soliman IBESandquarterlyearningspershare(epspiq). Excludethehighest andlowest1%ofvalues. FailureProbability Failureprobability Campbelletal 2008 1981 2003 Failureprobabilityis-9.16-.058*PRICE+.075*MB-2.13*CASHMTA -.045*RSIZE + 1.41*IdioRisk - 7.13*EXRETAVG + 1.42*TLMTA - 20.26*NIMTAAVG. PRICE is log(min(abs(prc), 15)); MB is shrout*abs(prc)/ceqq; CASHMTA is cheq/(shrout*abs(prc) + ltq); RSIZEislog(shrout*abs(prc)/sumofshrout*abs(prc)forthelargest 500companieseachmonth);IdioRiskisdefinedabove,EXRETAVG istheweightedaverageexcessreturn(log(1+ret)-log(1+mktrf)) overtheprevious12months,withweightonmontht-jbeing$\phiˆj$ and the sum scaled by $\frac1-\phi1-\phi1ˆ2$; TLMTA is total liabilities(ltq/(shrout*abs(prc)); NIMTAAVGisaweightedaverageof netincomeovertotalassets(ibq/(shrout*abs(prc)+ltq))overfour quarters,withweight$\phiqˆ$onquarter$t-q$andthesumscaled by$\frac1-\phi3ˆ1-\phi1ˆ2$.$\phi=2ˆ-\frac13$.Allinputvariablesare winsorizedatthe5thand95thpercentile.Excludeifpricelessthan1. FirmAgeMom FirmAge-Momentum Zhang 2004 1983 2001 6 month return, restricted to the bottom quintile of the crosssectionalfirmagedistribution. Excludeifpricelessthan5orfirm youngerthan12months. ForecastDispersion EPSForecastDispersion Diether,Malloyand 2002 1976 2000 Standarddeviationofearningsestimates(stdev\_est)scaledbymean Scherbina earningsestimate. GIndex GovernanceIndex Gompersetal 2003 1990 1999 Indexavailablefromhttp://faculty.som.yale.edu/andrewmetrick/data.html . Theindexisonlyavailableevery2-3yearsforeachfirm, wereplace intermediate missing values with the latest available one. Value-weighted. GrAdExp Growthinadvertisingexpenses Lou 2014 1974 2010 Logofadvertisingexpense(xad)minuslogofadvertisingexpenselast year.Excludeifpricelessthan5,xadlessthan.1orstockinthelowest decileofmarketvalueofequity. Continuedonnextpage 49

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description GrCAPX Changeincapex(twoyears) Andersonand 2006 1976 1999 Growthrateofcapitalexpenditures(capx)relativetotwoyearsago. Garcia-Feijoo Ifcapxismissing,replacewithannualchangeinproperty,plantand equipment(ppent). GrEmp Employmentgrowth Bazdresch,BeloandLin 2014 1965 2010 Changeinnumberofemployees(emp)betweent-1andt,scaledby averagenumberofemployeesint-1andt.Replacehirewith0ifemp orlaggedempismissing. GrGMToGrSales GrossMargingrowthoversales AbarbanellandBushee 1998 1974 1988 DefinegrossmarginGMasrevenue(sale)minuscostofgoodssold growth (cogs). GrGMToGrSalesisthepercentagegrowthofGMrelativeto averageGMinyearst-1andt-2,minusthepercentagegrowthofrevenuerelativetoaveragerevenueinyearst-1andt-2.Replacegrowth rateswithgrowthrelativetothepreviousyearonlyifdatafort-2are notavailable. GrLTNOA GrowthinLongtermnet Fairfieldetal 2003 1964 1993 Annualgrowthinnetoperatingassets,minusaccruals.Netoperating operatingassets assetsare(rect+invt+ppent+aco+intan+ao-ap-lco-lo)/at. Accrualsare(rect-l12.rect+invt-l12.invt+aco-l12.aco-(ap-l12.ap +lco-l12.lco)-dp)/((at+l12.at)/2) GrossProf grossprofits/totalassets Novy-Marx 2013 1963 2010 Revenue(sale)-costofgoodssolds(cogs), dividedby12months laggedtotalassets. GrSaleToGrInv Salesgrowthoverinventory AbarbanellandBushee 1998 1974 1988 Percentagegrowthinsales(sale)relativetoaveragesalesoft-1and growth t-2,minuspercentagegrowthininventory(invt)relativetoaverage inventoryoft-1andt-2.Bothgrowthtermsarecalculatedrelativeto t-1onlyift-2ismissing. GrSaleToGrOverhead Salesgrowthoveroverhead AbarbanellandBushee 1998 1974 1988 GrSaleToGrOverHead=Percentagegrowthinsales(sale)relativeto growth averagesalesoft-1andt-2,minuspercentagegrowthinadministrativeexpenses(xsga)relativetoaverageadministrativeexpensesoft-1 andt-2.Bothgrowthtermsarecalculatedrelativetot-1onlyift-2is missing. RemoveifinthehighestquintileofGrSaleToGrOverHead. Returnsarenicelymonotonicuntilthehighestquintile,consistent withoriginalpaper’srankregressions. Herf Industryconcentration HouandRobinson 2006 1963 2001 Three-yearrollingaverageofthethreedigitindustryHerfindahlin- (Herfindahl) dexbasedonfirmrevenue(sale).Excluderegulatedindustries(4011, 4210,4213&year$\leq$1980;4512&year$\leq$1978,4812,4813& year$\leq$1982,4900-4999inanyyear) High52 52weekhigh GeorgeandHwang 2004 1963 2001 Lettemphigh=price/bythemaximumdailypriceoverthepast twelvemonths. High52istherolling6monthaverageoftemphigh tosimulatetheoriginalpaper’s6-monthholdingperiods IdioRisk Idiosyncraticrisk Angetal 2006 1963 2000 StandarddeviationofresidualsfromCAPMregressionsusingthepast monthofdailydata.Valueweighted Illiquidity Amihud’silliquidity Amihud 2002 1964 1997 Past twelve month average of: daily return (abs(ret)) divided by turnover((abs(prc)*vol) Continuedonnextpage 50

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description IndIPO InitialPublicOfferings Ritter 1991 1975 1984 1 if IPO in the past 6-36 months. 0 otherwise. IPO dates are taken from Jay Ritter’s IPO data available at: http://bear.warrington.ufl.edu/ritter/ipodata.htm. Missing IPO datesimplyIndIPO=0 IndMom IndustryMomentum GrinblattandMoskowitz 1999 1963 1995 Weightedaverageoffirm-level6monthbuy-and-holdreturn. Averageistakenovertwodigitindustrieseachmonthandweightsare basedonmarketvalueofequity. IndRetBig Industryreturnofbigfirms Hou 2007 1972 2001 Averagemonthlyreturn(ret)ofthe30%largestcompaniesbymarketvalueofequityinthesameFama-French48industry. Exclude the largest 30% of companies for IndRetBig (not to compute the anomaly!) IntanBM Intangiblereturn DanielandTitman 2006 1968 2003 Ineachmonth,runacross-sectionalregressionofafirm’sfive-year stockreturnon5yearlaggedBM(definedabove)andaconstructed regressorthatisthechangeinBMfrom5yearsagototodayplusthe five-yearstockreturn.TheresidualfromthatregressionisIntanBM. IntanCFP Intangiblereturn DanielandTitman 2006 1968 2003 Ineachmonth,runacross-sectionalregressionofafirm’sfive-year stockreturnonthe5yearlaggedCFP=(netincome(ni)plusdepreciation(dp))/marketvalueofequityandaconstructedregressorthatis thechangeinCFPfrom5yearsagototodayplusthefive-yearstock return.TheresidualfromthatregressionisIntanCFP. IntanEP Intangiblereturn DanielandTitman 2006 1968 2003 Ineachmonth,runacross-sectionalregressionofafirm’sfive-year stockreturnonthe5yearlaggedEP=netincome(ni)/marketvalue ofequityandaconstructedregressorthatisthechangeinEPfrom5 yearsagototodayplusthefive-yearstockreturn. Theresidualfrom thatregressionisIntanEP. IntanSP Intangiblereturn DanielandTitman 2006 1968 2003 Ineachmonth,runacross-sectionalregressionofafirm’sfive-year stockreturnon5yearlaggedSP(definedabove)andaconstructed regressorthatisthechangeinSPfrom5yearsagototodayplusthe five-yearstockreturn.TheresidualfromthatregressionisIntanSP. IntMom IntermediateMomentum Novy-Marx 2012 1926 2010 Stockreturnbetweenmonthst-12andt-6 Investment Investment Titmanetal 2004 1973 1996 Ratioofcapitalinvestment(capx)torevenue(revt)dividedbythe firm-specific36-monthrollingmeanofthatratio.Excludeifrevenue lessthan\$10m. IO_ShortInterest InstitutionalOwnershipforstocks Asquith,Pathakand 2005 1980 2002 Exclude all stocks with short interest (ShortInterest) below .025. withhighshortinterest Ritter IO\_ShortInterestisinstitutionalownership(instown\_perc). Keep NYSEOnly. Continuedonnextpage 51

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description KZ KaplanZingalesindex Lamontetal 2001 1968 1997 -1.002* (net income (ni) + depreciation (dp))/total assets (at) + .283*(totalassets(at)+marketvalueofequity-bookvalueofequity (ceq) - deferred taxes (txdi))/total assets (at) + 3.319*(debt in currentliabilities(dlc)+long-termdebt(dltt))/(debtincurrentliabilities + long-term debt + book value of equity) - 39.368*(Dividends (divamt)/total assets) - 1.315*(cash and short-term investments(che)/totalassets).Replacetxdianddivamtwith0ifmissing. Leverage Marketleverage Bhandari 1988 1946 1981 Totalliabilities(lt)dividedbymarketvalueofequity. MaxRet Maximumreturnovermonth Balietal 2010 1962 2005 Maximumofdailyreturns(ret)overthepreviousmonth Mom12m Momentum(12month) JegadeeshandTitman 1993 1964 1989 Stockreturnbetweenmonthst-12andt-1. Mom1813 Momentum-Reversal DeBondtandThaler 1985 1933 1980 Stockreturnbetweenmonthst-18andt-13. Mom1m Shorttermreversal Jegedeesh 1989 1934 1987 Stockreturn(ret)overthepreviousmonth. Mom36m Long-runreversal DeBondtandThaler 1985 1926 1982 Stockreturnbetweenmonthst-36andt-13. Mom6m Momentum(6month) JegadeeshandTitman 1993 1964 1989 Stockreturnbetweenmonthst-6andt-1.Excludeifpricelessthan5. Mom6mJunk JunkStockMomentum Avramovetal 2007 1985 2003 Mom6m.Includeonlystockswithacreditrating(splticrm)ofBBBor lower MomRev MomentumandLTReversal ChanandKot 2006 1965 2001 Binaryvariableequalto1iffirmisinthehighestMom6mquintileand thelowestMom36mquintile,andequalto0iffirmisinthelowest Mom6mquintileandthehighestMom36mquintile.Excludeifprice lessthan5. MomVol MomentumandVolume LeeandSwaminathan 2000 1965 1995 Mom6m.Includeonlystocksinthehighestquintileofaveragetrading volume (vol) over the previous 6 months. Exclude NASDAQ stocks,ifpricelessthan1orifstockhasbeentradingforlessthan 24months. Mscore MohanramG-score Mohanram 2005 1978 2001 ExamineonlystocksinlowestBMquintile. Binaryvariablebased onsumofeightindicatorvariableswhichare: 1ifreturnonassets (ni/averageassets)abovethetwodigitindustrymedian;1inetcash flowtoassets(oancf/averageassets)abovethetwodigitindstrymedian;1ifnetcashflowgreaterthannetincome;1ifR&Dexpenseto assets(xrd/averageassets)greaterthantwodigitindustrymedian;1 ifcapitalexpenditure(capx/averageassets)greaterthantwodigitindustrymedian;1ifadvertisingexpenses(xad/averageassets)greater thantwodigitindustrymedian;1ifthevolatilityofnetincomeover thepast3yearsisbelowthetwodigitindustrymedian,1ifthevolatilityofrevenue(revt)overthepast3yearsisbelowthetwodigitindustrymedian. Thefinalvariableisequalto1ifthesumoftheabove8 indicatorsisgreaterthan5and0ifthesumislessthan2. NetDebtFinance Netdebtfinancing Bradshawetal 2006 1971 2000 Long-term debt issuance (dltis) minus long-term debt reduction (dltr)minuscurrentdebtchanges(dlcch),scaledbyaveragetotalassets(at)inyearst-1andt. Replacemissingvaluesofdlcchwith0. Excludeifratioisgreaterthan1. Continuedonnextpage 52

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description NetDebtPrice Netdebttoprice PenmanRichardson 2007 1961 2001 Long-termdebt(dltt)plusdebtincurrentliabilities(dlc)pluspre- Tuna ferredstock(pstk)pluspreferreddividendsinarrears(dvpa)minus treasurystock(tstkp)minuscashandshort-terminvestments(che), scaledbymarketvalueofequity. ExcludeifSICbetween6000and 6999,orifmissingvaluefortotalassets(at),netincome(ib),commonsharesoutstanding(csho),bookvalueofequity(ceq)orprice closefiscalyear(prcc\_f).Keeponly3rdB/MQuintile,followingTable4(andincontrasttoTable1). NetEquityFinance Netequityfinancing Bradshawetal 2006 1971 2000 Sale of common stock (sstk) minus purchase of common stock (prstkc),scaledbyaveragetotalassets(at)fromyearstandt-1. Excludeifabsolutevalueofratioisgreaterthan1. NetPayoutYield NetPayoutYield Boudoukhetal 2007 1984 2003 Dividends (dvc) plus purchase of common and preferred stock (prstkc)minussaleofcommonandpreferredstock(sstk),dividedby marketvalueofequity. NOA NetOperatingAssets Hirshleiferetal 2004 1964 2002 Differencebetweenoperatingassetsandoperatingliabilities,scaled bylaggedtotalassets. Operatingassetsaretotalassets(at)minus cash-andshort-terminvestments(che),operatingliabilitiesaretotal assetsminuslong-termdebt(dltt),minorityinterest(mib),deferred charges(dc)andbookequity(ceq). NumEarnIncrease Numberofconsecutiveearnings LohandWarachka 2012 1987 2009 Numberof4-quarternetincome(ibq)increasesovertheprevious2 increases years. OperLeverage OperatingLeverage Novy-Marx 2010 1963 2008 Sumofadministrativeexpenses(xsga)andcostofgoodssold(cogs), scaledbytotalassets(at).Usexsga=0ifxsgaismissing. OperProf operatingprofits/bookequity FamaandFrench 2006 1977 2003 Revenue(revt)minuscost(cogs)-administrativeexpenses(xsga)interestexpenses(xint),scaledbybookvalueofequity(ceq).Exclude smallestsizetercile. OptVol OptionVolumetoStockVolume JohnsonandSo 2012 1996 2010 Totalmonthlyoptionvolume(volume)overallputsandcalls,divided bymonthlystocktradingvolume(vol). Excludeifpricelessthan1 orsharecodegreater11oroptionvolumeorstockvolumedataare missingforthepreviousmonth. OrderBacklog Orderbacklog Rajgopaletal 2003 1981 1999 Orderbacklog(ob)dividedbyaveragetotalassets(at)inyearst-1and t.Excludeiforderbacklogis0. OrgCap OrganizationalCapital Eisfeldtand 2013 1970 2008 Defined recursively. Initialize with OrgCap = 4*general expenses Papanikolaou (xsga)inthefirstyear,andcalculateas.85*OrgCappreviousyear+ xsgacurrentyearthereafter.Scalebytotalassets(at). Continuedonnextpage 53

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description OScore OScore Dichev 1998 1981 1995 OScore=-1.32-.407*log(at/GNPdeflator)+6.03*(lt/at)-1.43*((actlct)/at)+.076*(lct/act)-1.72*I(lt>at)-2.37*(ib/at)-1.83*(fopt/lt)+ .285*(ib+ib$_t-12$+ib$_t-24$<0)-.521*((ib-ib$_t-12$)/(abs(ib) +.abs(ib$_t-12$))). fopt=oancfiffoptismissing. ExcludeExclude ifSICcodebetween3999and4999,orgreaterthan5999. Excludeif pricelessthan5.ThenexcludeifOScoreisinbottomquintileofOScore(originalpapershowsnon-monotonicreturns,asdoesourreplication) PayYield PayoutYield Boudoukhetal 2007 1984 2003 Sumofdividends(dvc), purchaseofcommonandpreferredstock (prstkc)andmax(preferredstockredemptionvalue(pstkrv),0),dividedbylag(marketvalueofequity,6months).ExcludeifPayoutYield $\leq$0. PctAcc PercentOperatingAccruals Hafzallaetal 2011 1989 2008 Incomebeforeextraordinaryitems(ib)minusnetcashflow(oancf) dividedbyabsolutevalueofib.Ifoancfismissing,PctAccisdefined as((act-act$_t-12$)-(che-che$_t-12$)-((lct-lct$_t-12$)-(dlcdlc$_t-12$)-(txp-txp$_t-12$)-dp))/abs(ib). Ineithercase,ifibis equalto0,divideby.01instead.Excludeifpricelessthan5. PctTotAcc PercentTotalAccruals Hafzallaetal 2011 1989 2008 Netincome(ni)minus(purchaseofcommonandpreferredstock (prstkcc)minussaleofcommonandpreferredstock(sstk)plusdividends(dvt),cashflowfromoperations(oancf),fromfinancing(fincf) andinvestment(ivncf)).Scaledbyabsolutevalueofnetincome. PensionFunding PensionFundingStatus FranzoniandMarin 2006 1980 2002 FR=(FVPA-PBO),scaledbymarketvalueofequity. FVPAispbnaa from1980to1986,pplao+pplaofrom1987to1997,andpplaoafter 1997. PBOispbnvvfrom1980to1986,pbpro+pbprufrom1987to 1997,andpbproafter1997.Excludeifpricelessthan5orshrcd>11. Price Price BlumeandHusic 1972 1932 1971 Logofabsolutevalueofprice(prc). PriceDelay Pricedelay HouandMoskowitz 2005 1964 2001 Regressdailystockreturn(ret)onmarketreturn(mktrf)in$t,t-1, \ldots,t-4$withobservationsoverthepreviousyear. Trimthehighestandlowest1%ofestimatedcoefficients.DefinePriceDelayasthe ratioof1*betaonmktrf$_t-1$+2*betaonmktrf$_t-2$+3*betaon mktrf$_t-3$+4*betaonmktrf$_t-4$,andbetaonmktrf$_t$+beta onmktrf$_t-1$+betaonmktrf$_t-2$+betaonmktrf$_t-3$+beta onmktrf$_t-4$.Thefinalvariableistheaverageofthatratiooverthe previousmonth. Profitability earnings/assets Balakrishnan,Bartovand 2010 1976 2005 Quarterly earnings per share (epspxq) times quarterly shares out- Faurel standingusedtocalculateEPS(cshprq)dividedbytotalassets(at). Excludeifpricelessthan1. ProfitMargin ProfitMargin Soliman 2008 1984 2002 Netincome(ni)overrevenue(revt).Excludeifpricelessthan5. Continuedonnextpage 54

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description Pscore PiotroskiF-score Piotroski 2000 1976 1996 Sumofnineindicatorvariableswhichare:1ifnetincome(ib)greater 0;1ifnetcashflow(oancf)greater0;1ifreturnonassets(ib/at)increasedrelativetopreviousyear;1ifnetcashflowgreaternetincome; 1iflong-termdebttoassets(dltt/at)declinedoverthepreviousyear; ifcurrentassetstocurrentliabilities(act/lct)increasedoverthepreviousyear;1ifebit/sale(ebit=ib+txt+xint)increasedoverthepreviousyear;1ifrevenuetoassetsincreasedoverthepreviousyear;1 ifshrout$\leq$shroutlastyear.Includehighestquintileofbook-tomarketonly.Excludeifmissinganyoftheinputvariables. RD R&Dovermarketcap Chanetal 2001 1975 1995 R&Dexpense(xrd)overmarketvalueofequity. RDIPO IPOandnoR&Dspending Gouetal 2006 1980 1995 Binaryvariableequalto1ifR&Dexpense(xrd)=0andIndIPO=1.0 otherwise. RDirtSurp Realdirtysurplus Landsman,Miller, 2011 1976 2003 DefineDirtySurplusasannualchangeinmarketablesecuritiesad- PeasnellandYeh justmentmsaplusannualchangeinretainedearningsadjustment (recta) + .65 times the annual change in min(Unrecognized prior servicecost(pcupsu)-Pensionadditionalminimumliability(paddml),0).Realdirtysurplusistheannualchangeinbookequity(ceq) minusdirtysurplusminus(netincome(ni)minusdividendspreferred (dvp)) + dividends (divamt) - end-of-fiscal-year-stock-price (prcc\_f)*annualchangeincommonsharesoutstanding(csho). RealEstate Realestateholdings Tuzel 2010 1971 2005 Industry-adjustedvalueofrealestateholdings. Realestateholdings arecalculatedas:PPEBuildingsatcost(fatb)plusPPELeasesatcost (fatl),dividedbyPPE(ppegt).Useppentifppegtismissing.Subtract monthlyindustry-meanatthe2digitSIClevel. RetConglomerate Conglomeratereturn CohenandLou 2012 1977 2009 Identify conglomerate firms as those with multiple OPSEG or BUSSEGentriesintheCompustatsegmentdata(andrequirethatat least80%offirm’stotalassetsarecoveredbysegmentdata).Compute monthlystockreturnatthe2-digitSIClevelforstand-alone(nonconglomerate)firmsonly,andmatchthosereturnstoconglomerates’ segments. Computeweightedconglomeratereturnastheindustry returnofstand-alonecompanies,weightedwithaconglomerate’stotalsalesineachindustry. RevenueSurprise RevenueSurprise JegadeeshandLivnat 2006 1987 2003 Define revenue per share as quarterly revenue (revtq) divided by quarterly common shares outstanding (cshprq). RevenueSurprise isthe4-quarterchangeinrevenuepershareminustheaverage4quarterchangeinrevenuepershareovertheprevious2years. RevenueSurpriseisscaledbyitsstandarddeviationovertheprevious2 years.Excludeifpricelessthan5. Continuedonnextpage 55

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description RIO_BM InstOwnandBM Nagel 2005 1980 2003 Residualinstitutionalownership(RIO)isdefinedaslog(institutional ownership (instown\_perc)/(1-institutional ownership)) + 23.6 - 2.89*log(marketvalueofequity)+.08*log(marketvalueofequity)$2ˆ$. Replaceinstown\_percwith0ifitismissing,with.9999ifit’sabove .9999,andwith.0001ifit’sbelow.0001.RIO\_BMisabinaryvariable equalto1ifafirmisinthehighestquintileofthemonthlyRIOdistributionandhasBMbelowthecross-sectionalmedian,and0ifafirm isinthelowestquintileofRIOandhasBMbelowthemedian. RIO_Disp InstOwnandForecastDispersion Nagel 2005 1980 2003 Binaryvariableequalto1ifRIO(definedabove)isinthehighestquintileandForecastDispersion(definedabove)isabovethemedian,0if RIOisinthelowestquintileandForecastDispersionisabovethemedian. RIO_IdioRisk InstOwnandIdioVol Nagel 2005 1980 2003 Binaryvariableequalto1ifRIO(definedabove)isinthehighestquintileandmonthlyIdioRisk(definedabove)isabovethemedian,0if RIOisinthelowestquintileandIdioRiskisabovethemedian. RIO_Turnover InstOwnandTurnover Nagel 2005 1980 2003 Binaryvariableequalto1ifRIO(definedabove)isinthehighestquintileandmonthlyturnover(vol/shrout)isabovethemedian,0ifRIO isinthelowestquintileandturnoverisabovethemedian. RoE netincome/bookequity HaugenandBaker 1996 1979 1993 Netincome(ni)overbookvalueofequity(ceq).Excludeifpriceless than5. SalesGr RevenueGrowthRank Lakonishoketal 1994 1968 1990 Rankfirmsbytheirannualrevenuegrowtheachyearoverthepast 5years. MeanRankRevGrowthistheweightedaverageofranksover the past 5 years, that is, MeanRankRevGrowth = (5*Rank$_t-1$ + 4*Rank$_t-2$+3*Rank$_t-3$+2*Rank$_t-4$+1*Rank$_t-5$)/15.ExcludeNASDAQstocks. SalesToPrice Sales-to-price Barbeeetal 1996 1979 1991 Ratioofannualsales(sale)tomarketvalueofequity. Seasonality ReturnSeasonality HestonandSadka 2008 1965 2002 Averagereturninthesamemonthoverthepreceding5years.Exclude NASDAQstocks. SEO PublicSeasonedEquityOfferings LoughranandRitter 1995 1975 1984 Binaryvariableequalto1ifseasonedequityofferingwithintheprevious12months.SEOdataarefromSDC. ShareIs1 Shareissuance(5year) DanielandTitman 2006 1968 2003 5-yeargrowthinnumberofshares.Numberofsharesiscalculatedas shrout/cfacshrtoadjustforsplits. ShareIs5 Shareissuance(1year) PontiffandWoodgate 2008 1970 2003 Growthinnumberofsharesbetweent-18andt-6.Numberofshares iscalculatedasshrout/cfacshrtoadjustforsplits. ShareRepurchase Sharerepurchases Ikenberry,Lakonishok 1995 1980 1990 Binaryvariableequalto1ifstockrepurchaseindicatedincashflow andVermaelen statement(prstkc>0),and0ifprstkc=0. ShareVol ShareVolume DatarNaikRadcliffe 1998 1962 1991 Sumofmonthlysharetradingvolume(vol)overthepreviousthree months, scaled by 3 times common shares outstanding (shrout). DropifShareVolisbelowitsmedian Continuedonnextpage 56

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description ShortInterest ShortInterest Dechowetal 2001 1976 1993 Short-interestfromCompustat(shortint)scaledbysharesoutstanding(shrout). Short-interestdataareavailablebi-weeklywithafour daylag.Weusethemid-monthobservationtomakesuredatawould beavailableinrealtime. SinStock SinStock(selectioncriteria) HongandKacperczyk 2009 1926 2006 UsingCompustatSegmentdata,sinAlgoisdefinedasabinaryvariableequalto1ifatleastonesegmentofafirmislistedasbeing in at least one of the following industries: sic $\geq$ 2100 & sic $\leq$2199,sic$\geq$2080&sic$\leq$2085,NAICSin\7132,71312, 713210,71329,713290,72112,721120\. Asintheoriginalpaper,we assumethatthesinstockindicatorappliestotheentirehistoryand futureoftheidentifiedfirm. sinAlgoisequalto0ifthefirmisnot identifiedintheCSSegmentdataasasinstockandifthefirmisin oneofthefollowingindustries:(sic$\geq$2000&sic$\leq$2046)OR (sic$\geq$2050&sic$\leq$2063)OR(sic$\geq$2070&sic$\leq$ 2079)OR(sic$\geq$2090&sic$\leq$2092)OR(sic$\geq$2095& sic$\leq$2099)OR(sic$\geq$2064&sic$\leq$2068)OR(sic$\geq$ 2086&sic$\leq$2087)OR(sic$\geq$920&sic$\leq$999)OR(sic $\geq$3650&sic$\leq$3652)ORsic==3732OR(sic$\geq$3931& sic$\leq$3932)OR(sic$\geq$3940&sic$\leq$3949)OR(sic$\geq$ 7800&sic$\leq$7833)OR(sic$\geq$7840&sic$\leq$7841)OR (sic$\geq$7900&sic$\leq$7911)OR(sic$\geq$7920&sic$\leq$ 7933)OR(sic$\geq$7940&sic$\leq$7949)ORsic$==$7980OR(sic $\geq$7990&sic$\leq$7999) Size Size Banz 1981 1926 1975 Logofmonthlymarketvalueofequity(abs(prc)*shrout)). Skew1 Volatilitysmirk Xing,ZhangandZhao 2010 1996 2005 UsingOptionMetricsdata,amongoptionswithdurationbetween10 and60days,impliedvolatilityofputoptionwithmoneynessclosest tobutabove1minusimpliedvolatilityofcalloptionwithmoneyness closesttobutbelow1. SmileSlope Slopeofsmile Yan 2011 1996 2005 UsingOptionMetricsdata,averageimpliedvolatilityofputoptions withdurationbetween15and30daysandroundeddeltaof-.5minus averageimpliedvolatilityofcalloptionswithdurationbetween15 and30daysandroundeddeltaof.5. Spinoff Spinoffs Cusatisetal 1993 1965 1988 SpinoffsareidentifiedasallobservationsintheCRSPacquisitionfile withvalidacpermentry. Spinoffisabinaryvariableequalto1ifa firmisidentifiedintheCRSPAcquisitiondataandifithasatmost oneyearofhistoryintheCRSPstockreturndata.Spinoffisequalto 0otherwise. StdTurnover Turnovervolatility ChordiaRoll 2001 1966 1995 Standarddeviationofturnover(vol/shrout)overthepast36months. Subrahmanyam Continuedonnextpage 57

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description SurpriseRD UnexpectedR&Dincrease Eberhartetal 2004 1974 2001 Binaryvariableequalto1if:R&D(xrd)scaledbyrevenue(revt)ispositive,R&Dscaledbytotalassets(at)ispositive,annualR&Dgrowth isgreaterthan5%,annualgrowthinR&Dovertotalassetsisgreater than5%.SurpriseRDis0otherwise. Tangibility Tangibility HahnandLee 2009 1973 2001 Cashandshort-terminvestments(che)plus.715*receivables(rect)+ .547*inventory(invt)+.535*property,plantandequipment(ppent), scaledbytotalassets(at).Onlydefinedformanufacturingfirms(SIC $\geq$2000andSIC<4000). Excludethelowesttercileofmanufacturingfirmsbytotalassets. Tax Taxableincometoincome LevandNissim 2004 1973 2000 RatioofTaxespaidandtaxshareofnetincome. Numeratorisdefinedasthesumofforeign(txfo)andfederal(txfed)incometaxes.If eitheroneismissing,numeratorisdefinedastotaltaxes(txt)minus deferredtaxes(txdi).Denominatoristheproductoftheprevailingtax rateandnetincome(ib).Taxrateis.48before1979,.46from1979to 1986,.4in1987,.34between1988and1992and.35from1993onwards.Ifnetincomeisnegative,andthenumeratorispositive,taxis definedas1.Excludeifpricelessthan5. UpForecast UpForecast BarberLehavy 2001 1985 1997 Binaryvariableequalto1ifmeananalystearningsforecastforthe MicNicholsTrueman nextquarter(meanest)hasimprovedoverthepreviousmonth,and0 otherwise. VarCF Cash-flowvariance HaugenandBaker 1996 1979 1993 Rollingvarianceof(ib+dp)/mve\_coverthepast60months(minimum24monthsdatarequired). VolMkt Volumetomarketequity HaugenandBaker 1996 1979 1993 Averagemonthlydollartradingvolume(vol*abs(prc))overtheprevious12months,scaledbymarketvalueofequity.Excludeifpriceless than5. VolSD VolumeVariance ChordiaRoll 2001 1966 1995 Rollingstandarddeviationofmonthlytradingvolume(vol)overthe Subrahmanyam past36months(requireatleast24observations).IncludeonlyNYSE stocks. VolumeTrend VolumeTrend HaugenandBaker 1996 1979 1993 Rollingcoefficientfromregressingmonthlytradingvolumeonalineartimetrendoverawindowof60months(requirethatatleast30 exist).Scalecoefficientby60-monthaverageoftradingvolume. XFIN Netexternalfinancing Bradshawetal 2006 1971 2000 Saleofcommonstock(sstk)minusdividends(dv)minuspurchase ofcommonstock(prstkc)pluslong-termdebtissuance(dltis)minus long-termdebtreductions(dltr).Scaledbytotalassets(at). ZeroTrade Dayswithzerotrades Liu 2006 1960 2003 Ineachmonth,countthenumberofdayswithnotrades.Definezerotradeasthenumberofdayswithouttradesplus(thesumofmonthly turnover(vol/shrout)dividedby48*10$5ˆ$),multipliedby21/number oftradingdayspermonth.Zerotradeisthe6-monthaverageofthat variable. Continuedonnextpage 58

TableA.1:(continued) Acronym Description Author(s) Pubyear SampleStart SampleEnd Description ZScore AltmanZ-Score Dichev 1998 1981 1995 1.2*(currentassets(act)-currentliabilities(lct))/totalassets(at)+ 1.4*(Retainedearnings(re)/totalassets(at))+3.3*(netincome(ni)+ interestexpense(xint)+totaltaxes(txt))/totalassets(at)+.6*(marketvalueofequity/Totalliabilities(lt))+revenue(revt)/totalassets (at).IncludeonlyNYSEstocks.ExcludeifSICcodebetween4000and 4999,orabove5999.ExcludeifZScoreisinbottomquintileofZScore (originalpapershowsnon-monotonicreturns,asdoesourreplication) 59

A.2. DetailsoftheMaximumLikelihoodEstimation Thelikelihoodisabittrickytowritedownasaresultofpublicationbias. The likelihoodofaobservingapair(r ,σ )needstobeconditionedonpublication: i i f r,σ|pub (r i ,σ i |pub i ,θ)= (cid:82) dσ˜ p (cid:163)(cid:82) (r d i / r˜ σ p i ( |θ r˜/ )f σ˜ r | | θ σ( ) r f i r | |σ σ ( i r˜ ,θ |σ˜ ) , f θ N µ ( ) lo (cid:164) g f N σ ( i l | o µ g σ σ˜ ,σ |µ σ σ ) ,σ σ) , (25) where, asbefore, f N and fτ arethenormalandscaledstudent’stdensities, and f r|σistheconditionaldensityofr|σ. f r|σisfoundbyconvolution (cid:90) f r|σ(r|σ,θ µ)= dµ˜fτ(µ˜|µ µ,σ µ,ν µ)f N (r|µ˜,σ). (26) The numerator of Equation (25) is intuitive: Due to publication bias, the likelihood of observing a pair r ,σ includes not only the densities of σ and r |σ , i i i i i but also the probability of passing the statistical requirements for publication p(r /σ |θ). Thedenominatorcomesfromthefactthat,sincesomeportfoliosare i i notpublished,weneedtorenormalizethedensityandmakesureitintegratesto 1. We evaluate the convolution in the numerator by standard numerical quadrature. The denominator of the likelihood involves three integrals, which istrickytodousingtraditionalmethods. Thus,wecomputethedenominatorby montecarlo. Another issue in the estimation is that the fat tail parameter ν µ has nonsmoothderivatives,whichtendstomakestandardoptimizersfail. Toovercome this problem, we optimize by iterating between a quasi-newton method for all parametersbesidesν µ,andusingamorerobustgoldensectionsearchbasedalgorithmforν µ. Theiterationstopswhenthelikelihoodstopsupdating. Wefind thisalgorithmtobequiterobust,andfaroutperformsstartingsimplexoptimizersatvariouspoints,forexample. Thelastissueisthatderivative-basedstandarderrorsmaynotworkwellwith thefattailparameter. Indeed,wefindthattheHessianstandarderrorunderestimatesuncertaintyinν µ insimulateddata. Toovercomethisissue,wecalculate standarderrorsbybootstrap. A.3. AdditionalEstimationFigures 60

FigureA.1:PairwiseCorrelations.Thishistogramshowsthedistributionofpairwisecorrelationsinourdatabaseofmonthlylong-shortportfolioreturns. 61

Figure A.2: Likelihood function. This figure plots the likelihood function near ourmaximumlikelihoodestimate(Table4). -140 -145 -150 -155 -160 -165 -0.5 0 0.5 µ µ ekil gol -140 -145 -150 -155 -160 ML estimate ML estimate -165 0.2 0.3 0.4 0.5 0.6 0.7 0.8 σ µ -140 -140 -145 -145 -150 -150 -155 -155 -160 -160 ML estimate ML estimate -165 -165 2 3 4 5 6 7 8 9 10 -2 -1.9 -1.8 -1.7 -1.6 -1.5 ν µ µ σ -140 -140 -145 -145 -150 -150 -155 -155 -160 -160 ML estimate ML estimate -165 -165 0.3 0.4 0.5 0.6 0.7 0.8 0.5 1 1.5 2 2.5 σ t σ cut -140 -145 -150 -155 -160 ML estimate -165 2 4 6 8 10 12 14 16 18 20 t slope 62

Figure A.3: Bootstrapped distribution of mean and median shrinkage. This figureplotsdetailsofthemeanshrinkagestandarderrorsinTable4. 0.1 0.08 0.06 0.04 0.02 0 0 5 10 15 20 25 Mean Shrinkage (%) ycneuqerF 0.1 0.08 0.06 0.04 0.02 0 0 5 10 15 20 25 Median Shrinkage (%) ycneuqerF 63

TableA.2:EstimationsunderAlternativeAssumptionsforµ µ Thistableshowsthatassumingalternativevaluesforthemeanoftruereturnsµ µ does not affect our headline shrinkage. We run the same estimation as in Table 4 column “all”butassumevariousvaluesofµ µ. “Baseline”reprintsthe“all”columnofTable4for comparison. The mean shrinkage is small regardless of the assumption for µ µ. An alternative definition for mean shrinkage that takes averages before taking ratios (mean µˆ i /meanr i )findssimilarshrinkageforallµ µassumptions. Thesimilarshrinkagehappens because the estimated σ µ decreases as one increases µ µ. Intuitively, if the averagetruereturnishigh,thenoneneedslittledispersiontomatchthepositivepublished truereturns. ThisrobustnessisconsistentwiththegeneralityofJamesandStein(1961) shrinkage,whichimprovesonthesamplemeanregardlessofthechoiceofµ µ. Theloglikelihoodsuggeststhatµ µclosetozeroisideal,however. Baseline Alternativeµ µ AssumedParameters µ µ meanoftruereturns 0.00 -0.40 -0.20 0.20 0.40 EstimatedParameters σ µ dispersionof 0.45 0.56 0.55 0.36 0.31 truereturns (0.05) (0.06) (0.06) (0.04) (0.04) ν µ fattail(d.o.f.)of 3.89 5.96 6.57 2.70 2.37 truereturns (1.38) (2.14) (1.92) (0.70) (0.39) µ σ meanoflog -1.67 -1.67 -1.67 -1.66 -1.68 standarderror (0.05) (0.04) (0.04) (0.05) (0.05) σ σ stdoflog 0.51 0.51 0.51 0.52 0.52 standarderror (0.02) (0.03) (0.03) (0.03) (0.03) t midpointof 1.61 1.62 1.61 1.60 1.56 cut t-statthreshold (0.08) (0.09) (0.07) (0.07) (0.06) t slopeof 10.97 10.90 11.09 11.02 11.24 slope t-statthreshold (41.78) (42.06) (41.96) (41.99) (42.77) EstimatedBiasAdjustments Meanµˆ /Meanr 12.68 13.24 12.52 12.03 9.58 i i (5.82) (5.63) (5.62) (5.48) (5.13) MeanShrinkage(%) 12.46 13.63 12.31 11.01 6.49 (1.75) (1.84) (1.85) (1.33) (0.84) StdShrinkage(%) 9.47 10.48 9.39 9.18 10.08 (1.13) (1.40) (1.31) (1.00) (1.19) Loglikelilhood -145.46 -145.56 -145.85 -146.22 -149.39 64

A.4. TheBenjamini-HochbergAdjustmentinOurModel TheBenjamini-Hochberg(BH)adjustmentrequiresveryfewassumptions. It merelyassumesthatacertain,unspecifiedproportionoft-statisticsarecloseto thenullN(0,1)distribution. Thisgeneralitycomesatacost,however. Withoutspecifyingtheproportion ofnullt-statistics,theadjustmentscanonlyprovideanupperboundonthefalse discoveryrate. Indeed,inmanycasestheBHadjustmentwillbeexcessivelyconservative,asweillustrateinthissection. To illustrate the mechanics of the BH adjustment, it helps to derive the adjustment within the context of our model. Suppose there is a small number ∆, suchthatanyportfoliowithµ ∈[−∆,∆]≈0. Let’slabeltheseportfoliosasnull . i i Theseareportfolioswithzerotruereturns,sotheirin-samplereturnsfollowthe traditional null distribution r |null ∼ N(0,σ ). This leads to a binary transfori i i mationofthemodelofSection3.1:   (cid:178) w/prob Pr(null ) i i t ∼ (27) i µ  i +(cid:178) otherwise σ i i Considerthet-stathurdlet . Forportfolioswhichmeetthishurdle,thefalse h discoveryrateis Pr(t >t |null )Pr(null ) (1−Φ(t ))Pr(null ) Pr(null |t >t )= i h i i = i i . (28) i i h Pr(t >t ) Pr(t >t ) i h i h Where Φ() is the standard normal CDF. Note that (1−Φ(t )) = p , the p-value i i corresponding to t . Also, the denominator can be estimated using its sample i counterpart(assumingallnarrativeportfoliosareobserved). Thesefactsleadto theBHadjustment Pr(null )p Pr(null |t >t )= i h (29) i i h Proportionofportfolioswitht >t i h p ≤ h . (30) Proportionofportfolioswitht >t i h Thus, BH is an upper bound, rather than a direct estimate of the false discoveryrate. Moreover,theBHadjustmentisexcessivelyconservativeifPr(null ) i is far from 1. For example, if null portfolios comprise roughly half the data (as in our estimation and in Harvey, Liu, and Zhu (2015)), then the BH FDR bound 65

exceedstheactualFDRbyafactorof2. ThenullhypothesisdiscussedinSection6.1µ ≤0cannotbeexaminedusing i BH’salgorithmwithouttheadditionalestimationofthedistributionofµ .Tosee i this,notethatthefalsediscoveryrateforµ ≤0is i Pr(t >t |µ ≤0)Pr(µ ≤0) Pr(null |t >t )= i h i i (31) i i h Pr(t >t ) i h (cid:90) 0 (cid:183) (cid:181) µ (cid:182)(cid:184) Pr(µ ≤0) = −∞ dµfµ(µ|θ) 1−Φ t i − σ i i Pr(t i i >t h ) . (32) where fµ(µ|θ)isthedistributionoftruemeansthatweestimateinSection??. A.5. MultipleTestsoftheNull: Bias-Adjustedt-stat<1.96 The low t-stat hurdles in Section 6.1 are due to the inadequacy of the traditional null hypothesis of µ =0. This null describes only a tiny portion of nari rative predictors. As a result, the null is ineffective for isolating cases worthy of furtherstudy. When the traditional null is a poor fit, one may want to use an empirical null, thatis, anullwhichisdesignedtogenerateunusualandinterestingcases. Thisnotionofestimatinganulldistributionisnotpossibleinclassicalsingletest statistics,butiscommoninlarge-scalestudies(Efron(2012)). Inthissection,weexamineanullhypothesiswhicheffectivelygeneratesinterestingpredictors. Specifically,wedefineanullpredictorasonethatsatisfies truereturn truet-stat≡ <1.96. (33) standarderror Thisnullismotivatedbyboththeoryanddata. Fromatheoreticalperspective,Equation(33)isanaturalextensionofthetraditionalt-stat<1.96hurdle. As the observed t-stat is a noisy estimate of the true t-stat, roughly half of the true t-statswillbebelowtheobservedone. UsingthenullinEquation(33)limitsthis uncertainty, and provides a higher order assurance that the true t-stat exceeds 1.96. From an empirical perspective, the data show that we need a rather strict definition of a null in order to isolate unusual cases. As we will see, relatively few narrative portfolios satisfy equation (33), and those that do are likely to be portfoliosworthyoffurtherresearch. 66

FigureA.4illustratestheFDRimpliedbythenull(33). Thetoppanelshowsa scatterplotofpublishedtruet-statsagainstobservedt-statsfromsimulatingthe estimatedmodel. Iftherewasnopublicationbias,observedt-statswouldbean unbiasedestimateofthetruet-stat,andthescatterplotswouldbeevenlyspread acrossthe45degreeline(dottedline). Thereisabitofpublicationbias,andthus therearemoremarkersbelowthe45degreelinethanaboveit. Figure A.4: Multiple Tests of the Null: True t-stat < 1.96. We simulate narrative portfolios using our estimated model (Table 4). The top panel shows a scatter plot of true t-stats against observed t-stats, where true t-stat = [true return]/[standard error]. Non-null predictors are those with true t-stats > 1.96 (light dots). The false discovery rate for a given t hurdle is the fraction of predictorswhichexceedthehurdlethatarenull. 8 6 4 2 0 0 1 2 3 4 5 6 7 8 Observed t-stat tats-t eurT Null (false discovery) t hurdle: FDR = 5% t hurdle: FDR = 1% 60 50 40 30 20 10 0 0 1 2 3 4 5 6 7 8 t-stat Hurdle )%( etaR yrevocsiD eslaF Naive t hurdle = 1.96 t hurdle: false discoveries = 5% t hurdle: false discoveries = 1% Despitethefactthatthebiasadjustmentsaresmall,manypredictorsarenull (reddots). Thepresenceofmanynullpredictorsisduetothestringencyofour nulldefinition. Bydesign,onlyabouthalfofthepredictorswithobservedt-stats around2are“significant.” The bottom panel shows the FDR as a function of the t-stat hurdle. Using a 67

hurdle of 0, 54% of predictors are null, and roughly 20% of predictors are null usingthetraditionalhurdleof1.96. It’snotuntilt-stathurdlesabove3.0thatone achievesanFDRrecommendedbyHLZ.Indeed,ahight-statof3.92isrequired toachieveanFDRof1%. Thet-stathurdleof3.92effectivelygeneratesinterestingacademiccasestudies.Predictorsthatmeetthishurdleareverylikelytobenotableinthetraditional academicsense. Asthenumberofpredictorsavailableforstudyhasbecomeunwieldy,thishigherhurdlemaybehelpfulforfocusingtheliterature. 68

References Alti, Aydog˘an and Sheridan Titman. “A Dynamic Model of Characteristic-Based ReturnPredictability”(2017). Amihud, Yakov. “Illiquidity and stock returns: cross-section and time-series effects”.Journaloffinancialmarkets5.1(2002),pp.31–56. Andrews,IsaiahandMaximilianKasy.Identificationofandcorrectionforpublicationbias.Tech.rep.NationalBureauofEconomicResearch,2017. Ang, Andrew, Robert J Hodrick, Yuhang Xing, and Xiaoyan Zhang. “The crosssectionofvolatilityandexpectedreturns”.TheJournalofFinance61.1(2006), pp.259–299. Benjamini, Yoav and Yosef Hochberg. “Controlling the false discovery rate: a practicalandpowerfulapproachtomultipletesting”.Journaloftheroyalstatisticalsociety.SeriesB(Methodological)(1995),pp.289–300. Black, Fischer. “Beta and return”. The Journal of Portfolio Management 20.1 (1993),pp.8–18. Chen, Andrew Y and Mihail Velikov. “Accounting for the Anomaly Zoo: A Trading Cost Perspective”. Available at SSRN: https://papers.ssrn.com/abstract=3073681(2017). Chordia,Tarun,AmitGoyal,andAlessioSaretto.“p-hacking:Evidencefromtwo mmilliontradingstrategies”(2017). Christensen,GarretSandEdwardMiguel.Transparency,Reproducibility,andthe Credibility of Economics Research. Tech. rep. National Bureau of Economic Research,2016. Cochrane,JohnH.“Presidentialaddress:Discountrates”.TheJournalofFinance 66.4(2011),pp.1047–1108. Cusatis,PatrickJ,JamesAMiles,andJRandallWoolridge.“Restructuringthrough spinoffs: The stock market evidence”. Journal of Financial Economics 33.3 (1993),pp.293–311. Dawid, AP. “Selection paradoxes of Bayesian inference”. Lecture Notes- MonographSeries(1994),pp.211–220. Efron, Bradley. Large-scale inference: empirical Bayes methods for estimation, testing,andprediction.Vol.1.CambridgeUniversityPress,2012. — “Tweedie’sformulaandselectionbias”.JournaloftheAmericanStatisticalAssociation106.496(2011),pp.1602–1614. 69

Feng, Guanhao, Stefano Giglio, and Dacheng Xiu. “Taming the Factor Zoo” (2017). Gompers,Paul,JoyIshii,andAndrewMetrick.“Corporategovernanceandequity prices”.Thequarterlyjournalofeconomics118.1(2003),pp.107–156. Green, Jeremiah, John RM Hand, and Frank Zhang. “The remarkable multidimensionalityinthecross-sectionofexpectedUSstockreturns”.Availableat SSRN2262374(2014). Harvey, Campbell R. “Reflections on Editing the Journal of Finance, 2006 to 2012”.SecretsofEconomicsEditors(2014),p.67. Harvey, Campbell R, Yan Liu, and Heqing Zhu. “... And the cross-section of expectedreturns”.ReviewofFinancialStudies29.1(2015),pp.5–68. Haugen,RobertAandNardinLBaker.“Commonalityinthedeterminantsofexpected stock returns”. Journal of Financial Economics 41.3 (1996), pp. 401– 439. Hedges,LarryV.“Modelingpublicationselectioneffectsinmeta-analysis”.StatisticalScience(1992),pp.246–255. Hong, Harrison and Marcin Kacperczyk. “The price of sin: The effects of social normsonmarkets”.JournalofFinancialEconomics93.1(2009),pp.15–36. Hou,Kewei,ChenXue,andLuZhang.ReplicatingAnomalies.Tech.rep.National BureauofEconomicResearch,2017. Ikenberry, David, Josef Lakonishok, and Theo Vermaelen. “Market underreactiontoopenmarketsharerepurchases”.Journaloffinancialeconomics39.2-3 (1995),pp.181–208. Jacobs,HeikoandSebastianMüller.“AnomaliesAcrosstheGlobe:OncePublic, NoLongerExistent?”(2016). James,WilliamandCharlesStein.“Estimationwithquadraticloss”.Proceedings ofthefourthBerkeleysymposiumonmathematicalstatisticsandprobability. Vol.1.1961,pp.361–379. Jensen,MichaelCandGeorgeABennington.“Randomwalksandtechnicaltheories:Someadditionalevidence”.JournalofFinance25(1970),pp.469–482. Kozak,Serhiy,StefanNagel,andShrihariSantosh.“ShrinkingtheCrossSection” (2017). LaPorta,Rafael.“Expectationsandthecross-sectionofstockreturns”.TheJournalofFinance51.5(1996),pp.1715–1742. Linnainmaa, Juhani T and Michael R Roberts. The history of the cross section of stockreturns.Tech.rep.NationalBureauofEconomicResearch,2016. 70

Liu, Laura,HyungsikRogerMoon,andFrankSchorfheide. ForecastingwithDynamicPanelDataModels.Tech.rep.WorkingPaper,2016. Liu,Qi,LeiLu,BoSun,andHongjunYan.“Amodelofanomalydiscovery”.AvailableatSSRN2431498(2015). Lo,AndrewWandACraigMacKinlay.“Data-snoopingbiasesintestsoffinancial assetpricingmodels”.Reviewoffinancialstudies3.3(1990),pp.431–467. Marquering, Wessel, Johan Nisser, and Toni Valla. “Disappearing anomalies: a dynamic analysis of the persistence of anomalies”. Applied Financial Economics16.4(2006),pp.291–302. McLean,RDavidandJeffreyPontiff.“Doesacademicresearchdestroystockreturnpredictability?”TheJournalofFinance71.1(2016),pp.5–32. Merton,RobertC.Onthecurrentstateofthestockmarketrationalityhypothesis. 1987. Penasse,Julien.“UnderstandingAnomalyDecay”(2017). Ritter,JayR.“Thelong-runperformanceofinitialpublicofferings”.Thejournal offinance46.1(1991),pp.3–27. Senn,Stephen.“Anoteconcerningaselection“paradox”ofdawid’s”.TheAmericanStatistician62.3(2008),pp.206–210. Sullivan,Ryan,AllanTimmermann,andHalbertWhite.“Dangersofdatamining: The case of calendar effects in stock returns”. Journal of Econometrics 105.1 (2001),pp.249–286. — “Data-snooping,technicaltradingruleperformance,andthebootstrap”.The JournalofFinance54.5(1999),pp.1647–1691. Titman, Sheridan, KC John Wei, and Feixue Xie. “Capital investments and stock returns”.JournaloffinancialandQuantitativeAnalysis39.4(2004),pp.677– 700. Yan,XueminSterlingandLinglingZheng.“FundamentalAnalysisandtheCross- SectionofStockReturns:AData-MiningApproach”.TheReviewofFinancial Studies30.4(2017),pp.1382–1423. 71