Gone are the days of red ink-stained editors straining their eyes over a slush pile, a pack of Marlboros and a glass of cheap scotch their only company in the dim, slatted light of their tiny office. Today, editorial acquisitions are increasingly driven by sales data. Editors look at the number of books the author has previously sold, as well as sales of comparable titles, before making the decision to acquire a book. But are these numbers telling the whole story? Trade fiction publishers can learn from scholarly presses, who are developing far more nuanced metrics to rank article, author, and journal impact. This paper will examine these metrics, then explore the challenges and opportunities for trade publishers in building similar rankings for themselves.
Scholars receive funding from different sources, so there is a strong need to measure and evaluate the impact of their research. Bibliometrics, “the statistical analysis of books, articles, or other publications,”1 is one way of evaluating impact. The most common metric used is citation count, in which often-cited articles are ranked more highly than those with lower citation counts. Different indices, such as the h-index2 and the Journal Impact Factor3 have been developed to measure impact through citations. These metrics, however, present several problems. The first is that they’re slow: citations can take years to accrue, making it difficult to accurately measure recent work 4. Secondly, measures like JIF are proprietary5 and easily manipulated,6 making them difficult to rely on.7 Thirdly, these measures don’t take into account other ways in which academic work can be shared or engaged with. Not all work is published in article form, and articles can generate conversations in other ways besides citation.
To counter this, organizations such as Altmetric.com8 have developed altmetrics, or article-level metrics. These nuanced measurements take into account a variety of different article features, including blog, Wikipedia, Twitter and Facebook mentions, article downloads, mainstream media citations, StackExchange and other forum posts, and more. What characterizes different altmetrics is that they draw on disparate sources created by a wide user base to evaluate articles.
And it works. Studies have shown that altmetrics can predict citation rates, but at a more rapid speed.9 10 11 Of course, these studies still privilege traditional citation metrics by using them as benchmarks of success. In fact, the real power of altmetrics is in their ability to identify different ways of measuring the impact of a text. “Altmetrics are fast, using public APIs to gather data in days or weeks. They’re open–not just the data, but the scripts and algorithms that collect and interpret it,” Jason Priem writes in “Altmetrics: A Manifesto.”12 But more importantly, he adds, their diversity makes them better able to navigate an academic landscape which now includes raw data sets, tweets, blogs, and other items in addition to traditional articles.13 In addition, altmetrics have the ability to emphasize other forms of impact: the ability of a work to provoke discussion, to cross over between disciplines, or to inform students and non-academic readers.
Fiction publishers and acquisition assessment
Fiction publishers invest in their authors in much the same way universities invest in academics. Publishers take on the financial risk of printing a book and paying an author advance. Only too often, they have no idea whether their risk will pay off. Trade publishers live in what Tom Davenport calls a “disadvantaged” industry: a B2B2C industry in which retailers, such as Amazon and Indigo, hold all the data about the end customers. 14. In the past, acquisitions were made blindly, based on gut instinct or an editor’s sense of the market. “It was difficult to discern sales patterns to see how well or poorly a book was doing until much later—sometimes months later—when the publishers receive returns,” Amanda Regan writes in her report on sales data. 15 Publishers were taking huge financial risks on authors without knowing if they would be successful. Even when a publisher’s instinct paid off, the process was still problematic: when acquisitions editors say that books select themselves, what they are actually saying is that they choose books within a framework of values they see no need to question.’ 16 By ignoring data, publishers were potentially ignoring larger cultural trends about what people wanted to read, acting on the assumption that the public was just like them.
In 2005, BookNet Canada launched its SalesData service,17 which provides subscribers with week-to-week aggregated sales data for any ISBN. Since then, publishers have shifted to acquiring books based, at least in part, on sales data: that of an author’s past works, or of comparable titles But these metrics come with their own set of problems. First of all, they are overly simplistic: publishers often use a single data point, total sales, as a predictor of future success. Secondly, they are subjective: comparable titles are still picked entirely according to editorial discretion. Finally, they favour mass-market bestsellers, since higher sales figures simply indicate broad appeal. For most books, however, a more successful strategy is to market directly to a niche audience who will engage with the content. To compete for attention with data-rich companies such as Facebook and Google, Davenport argues, “editing and editorial decision-making will have to become data-driven. Social media will have to be mined for sentiment along with content clickstream data. Publishers will have to compile insights on what really works, combining data analytics with knowledge management.”18 Acquisitions editors need more sophisticated metrics if they are to properly assess a work’s impact.
Altmetrics for fiction publishers
Fiction publishers can develop more nuanced metrics, similar to altmetrics used by academic publishers. One of the major components of altmetrics is online engagement, and this feature translates well to trade book publishers. Publishers should track mentions of a book or author on every social media platform, including Twitter, Facebook, Goodreads, and more. They should also quantify mentions in blogs, articles, and other media. There have been a few attempts to measure social media sentiment towards academic articles.19 Trade publishers may wish to expand upon these attempts and track sentiment across social media, since readers will be unlikely to pick up a book that has received overwhelmingly negative reviews.
Publishers could also track fan fiction online as a measure of strong engagement: “Within publishing, these writers represent the kind of ‘prosumer’ audience that has broadened the market for things like digital cameras, home theater and more. Online, there is a ‘flood of amateur collaboration’ we can embrace and benefit from,” Brian O’Leary writes.20 This includes not only text-based fan fiction but videos, photos, songs and art posted online that reference the work. A more ambitious project for publishers may be to track searches for keywords which appear in a book. If many people look up, in order, an obscure word used in Chapter 1, a song reference in Chapter 2, and a movie referenced in Chapter 3, they are engaging with the text across media in a way that is valuable to the publisher.
As well as measuring transmedia engagement, publishers can measure intertextual influence. Although by convention fiction books don’t cite each other, they do exist in a network of influence. Fifty Shades of Grey author E.L. James owes a debt to Stephenie Meyer, and most Western fiction owes a debt to Shakespeare. Is it possible to trace this influence across fiction? In the most traditional sense, publishers could count quotations from their books in other works, but this would likely only impact a very few well-known books. So far only a few studies have attempted to track an author’s influence on other texts. One, conducted by the Stanford Literary Lab, mapped David Foster Wallace’s works within a network of texts by extracting mentions of other books and authors from Amazon and mass media reviews. “In recommendation networks, the more times a text is recommended ‘by’ another text, the higher its prestige value. In review networks, where the links (based on co-occurrences) have no directionality, it is even simpler: nodes with the most links are the most prestigious,” Ed Finn explains.21 The study found that Amazon reviewers situated Wallace in a richer and more diverse network of texts than mass media reviewers did. While his method needs some work (both “Wallace is the next Shakespeare” and “Shakespeare, he ain’t” connect the author to Shakespeare), his approach is interesting. Publishers could use similar techniques to map texts in a network using Amazon reviews, Goodreads bookshelves and other user-generated data. This would help them not only to better measure impact, but also to better position and market books.
The above metrics assess the impact of a book after it hits the market to measure author influence. But is there a way to measure the future influence of first-time authors? In his 2012 study, Rui Yan found 11 features of articles and mapped them against citation counts. Three of these features were based solely on the content of the article The first, novelty, measured the novelty of the statements in the article. Yan found that citation count increased with novelty up to a point, and began to decrease after a certain threshold. This showed, he argued, that works which strayed too far from the norm were less likely to be widely cited. The second feature, topic rank, measured the popularity of the subject, and correlated with citation count. The third, diversity, measured the amount of topics in the article and found that in general, citation count increased with diversity. 22 Trade publishers could develop similar factors for text novelty, genre, and subject, and test them accordingly, to help them assess incoming manuscripts.
Some may argue against evaluating authors’ and books’ impacts at all. They may point to the difficulty of quantifying artistic merit, or the problem of identifying talent which may only be recognized years down the line. These concerns are valid, and I am not suggesting that impact metrics should be the only consideration editors use to drive decisions. The purpose of these measures, as with academic altmetrics, is not solely to inform acquisitions. Instead, they fill two other necessary functions.
First, they filter existing published content to help it get into the hands of the right readers. In her book Planned Obsolescence, Kathleen Fitzpatrick develops a framework for how reviews can help filter academic articles after publication. She talks about establishing a “trust metric” that will rank authors based on their reputation and authority within the community.23 The same concept applies to trade publishing, where the marketing department works hard to establish an author’s authority through jacket copy, interviews, and more. With an impact metric for both the book and the author, readers can find titles they trust within an over-saturated market. These metrics could be customized to each user’s tastes, like the Amazon “Recommended for you” feature. They could even be tweakable by the user, allowing them to emphasize different features of the model (i.e. choosing to rank social media mentions more highly than mass media mentions or vice versa).24
Second, metrics provide authors with an assessment of their own impact which can translate into other, indirect benefits. In his book The Long Tail, Chris Anderson writes about “The Reputation Economy: “Down in the tail, where distribution and production costs are low (thanks to the digital technologies), business considerations are often secondary. Instead, people create for a variety of other reasons — expression, fun, experimentation, and so on. The reason one might call it an economy at all is that there is a coin of the realm that can be every bit as motivating as money: reputation. Measured by the amount of attention a product attracts, reputation can be converted into other things of value: tenure, audiences, and lucrative offers of all sorts.” 25. Most small- and mid-list authors currently reside in the long tail. They often do not earn enough from advances or royalties to support themselves, but choose to write for other reasons. However, with trusted, recognized measures of their impact, they could turn their book into a better job, a speaking engagement, or a more profitable contract, just as an academic leverages high metrics into tenure, promotion, or increased funding.
I have tried to limit my analysis to forms of data that are already accessible to publishers: social media, reviews, and the manuscript itself. However, publishers need to demand access to data from retailers such as Amazon and Kobo. This data should include reader engagement with purchased books (such as time spent reading and completion rate); references to their own titles in works published by other presses (including indirect mentions and quotations), and data about consumer buying habits (including networks of books bought by readers of their book). With this information publishers could develop even better author metrics, and compensate for the fact that far fewer trade books are accessible online for free or through subscription services as compared to academic books and articles.
As trade publishers reevaluate their metrics, though, this lack of accessibility may change. Trade publishers who adopt an altmetric-like model will need to be aware that their own impact as a press is measureable too. This provides an opportunity for them to define their brand in their reader’s eyes. But it also leads to increased competition, not for buyers, but for attention. To succeed, publishers will need to learn to value reader’s engagement on its own terms, rather than as a direct lead-in to sales. They will have to make sure their texts are easily shareable and clippable, and use the data they gather to inform marketing and production as well as acquisitions.
1. “Bibliometrics Definition.” OECD Glossary of Statistical Terms. Accessed February 27, 2015. http://stats.oecd.org/glossary/detail.asp?ID=198.
2. Hirsch, J. E. “An Index to Quantify an Individual’s Scientific Research Output.” Proceedings of the National Academy of Sciences 102, no. 46 (November 15, 2005): 16569–72. doi:10.1073/pnas.0507655102.
3. “Journal Impact Factor.” Journal Impact Factor. Accessed February 27, 2015. http://jifactor.com/.
4 Thelwall, Mike, Stefanie Haustein, Vincent Larivière, and Cassidy R. Sugimoto. “Do Altmetrics Work? Twitter and Ten Other Social Web Services.” Edited by Lutz Bornmann. PLoS ONE 8, no. 5 (May 28, 2013): e64841. doi:10.1371/journal.pone.0064841.
5. Rossner, M., H. Van Epps, and E. Hill. “Show Me the Data.” The Journal of Cell Biology 179, no. 6 (December 17, 2007): 1091–92. doi:10.1083/jcb.200711140.
6. The PLoS Medicine Editors. “The Impact Factor Game.” PLoS Medicine 3, no. 6 (2006): e291. doi:10.1371/journal.pmed.0030291.
7. Priem, Jason, Dario Taraborelli, Paul Groth, and Cameron Neylon. “Altmetrics: A Manifesto,” 2010. http://altmetrics.org/manifesto/.
8. “What Does Altmetric Do?” Altmetric. Accessed February 27, 2015. http://www.altmetric.com/whatwedo.php.
9. Eysenbach, Gunther. “Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact.” Edited by Anne Federer. Journal of Medical Internet Research 13, no. 4 (2011): e123. doi:10.2196/jmir.2012.
10. Thelwall, “Do Altmetrics Work?”
11. >Yan, Rui, Congrui Huang, Jie Tang, Yan Zhang, and Xiaoming Li. “To Better Stand on the Shoulder of Giants.” In Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries, 51–60. ACM, 2012. doi:10.1145/2232817.2232831.
12. Priem, “Altmetrics: A Manifesto.”
13. Priem, “Altmetrics: A Manifesto.”
14. Davenport, Tom. “Book Publishing ’s Big Data Future.” Harvard Business Review, March 2014. https://hbr.org/2014/03/book-publishings-big-data-future/.
15. >Regan, Amanda Jia’en. Data-Driven Publishing: Using Sell-through Data as a Tool for Editorial Strategy and Developing Long-Term Bestsellers. MPub Project Report. Vancouver, BC: Simon Fraser University, Spring 2012. http://publishing.sfu.ca/?p=1915&preview=true
16. Curran, J., qtd. in C. Clayton Childress. “Decision-Making, Market Logic and the Rating Mindset: Negotiating BookScan in the Field of US Trade Publishing.” European Journal of Cultural Studies 15, no. 5 (2012): 604–20. http://ecs.sagepub.com/content/15/5/604
17. “About SalesData.” BookNet Canada. Accessed February 27, 2015. http://www.booknetcanada.ca/salesdata/.
18. Davenport, “Book Publishing’s Big Data Future.”
19. Thelwall, Mike, Andrew Tsou, Scott Weingart, Kim Holmberg, and Stefanie Haustein. “Tweeting Links to Academic Articles.” Cybermetrics: International Journal of Scientometrics, Informetrics and Bibliometrics, no. 17 (2013): 1–8. http://dialnet.unirioja.es/servlet/revista?codigo=5578
20. O’Leary, Brian. From Competitors to Collaborators : 12 Steps for Publishers in the Digital Age, 2014. http://www.magellanmediapartners.com/publishing-innovation/12-steps-for-publishers-in-the-digital-age/.
21. Finn, Ed. “Becoming Yourself: The Afterlife of Reception,” 2011. Stanford Literary Lab Pamphlets. litlab.stanford.edu/LiteraryLabPamphlet3.pdf.
22. Yan, “To Better Stand on the Shoulders of Giants.”
23. Fitzpatrick, Kathleen. Planned Obsolescence: Publishing, Technology, and the Future of the Academy. NYU Press, 2011. http://www.plannedobsolescence.net/about/.
24. Davis, Phil. “Visualizing Article Performance-Altmetric Searches for Appropriate Display.” The Scholarly Kitchen, September 30, 2013. http://scholarlykitchen.sspnet.org/2013/09/30/visualizing-article-performance-altmetrics-searches-for-appropriate-display/.
25. Anderson, Chris. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, 2006. http://www.thelongtail.com/about.html