How to Write a Bestseller: What can big data teach us about writing the novel?

If there is one thing that I have learned about creative writing it’s that there are no short-cuts. Yes, one can learn the tools of the trade and this might help us organise our thoughts and produce better work. Yes, there are certain rules that should be followed (which, as we know, are made to be broken but one needs to spend time learning them first right?). Yes, books about writing books are so numerous they practically constitute their own genre but none of this makes the actual task easier or faster.

Despite our continued quest to find the ‘secrets’ to writing a bestseller there is one, solitary truism – it takes hard work. Lots of it. And even at that there are no guarantees. It seems that those who find the holy grail of publishing have had novel-writing fairy dust sprinkled liberally on their projects and that success requires a little bit of magic in addition to blood sweat and tears.

Perhaps it is unsurprising then, that in the age of big data, scientists have tried to analyse ‘the bestseller’ and discover what it is about them that makes everybody tick. Rather than the anecdotal ‘how to’s’ from the likes of Stephen King, the latest developments promise a scientific approach, identifying algorithms to unlock the key to writerly fame and fortune.

The Bestseller Code, is the latest in a long line of ‘how to write a novel’ books, which promise to divulge the secrets to writing a hit but uses big data to deconstruct the DNA of the blockbuster novel. (Disclaimer: I have not yet read the book so cannot comment on the value of it’s advice. What follows are just my own thoughts on the relationship between what we might call literary science and art.) 

The book is the culmination of 5 years of text-mining research on 20,000 novels on the New York Times bestseller list, which identified the key elements of a successful novel. The authors Jodie Archer and Matthew Jockers argue, for example, that bestsellers are more likely to contain human characters than non-human ones like unicorns and dwarves, and that words such as “need” “love” and “miss” appear far more often. They claim, based on their work, that they can predict a bestseller with 80% accuracy.

I’ll admit, I’m intrigued by the idea. I like science, numbers and big data. I am not a literary type who is afraid of anything that can’t be counted on ones fingers. Yet, I am dubious about how useful it is to anyone actually trying to write a novel. One of the problems, I feel, is that the purpose of scientific discovery is to generalise. When the results of scientific research shows that a phenomenon (with a certain degree of confidence) works for ‘all things’ – that for every time we see X, Y happens – science, is doing what it is supposed to. Art, by contrast does the opposite. It seeks to find the sweet spot in a person’s emotions, which tries to take the general and make it personal.

So a study, which tries to generalise the artistic begs the question of what purpose it serves.  If we seek to reduce novels/bestsellers to a formula then do we expect, in the future, that novels will be written by Artificial Intelligence (AI)? Is the point of investigating the science behind the novel supposed to help writers write more efficiently by setting down new ‘rules’? I don’t believe for a minute that science is so threatened by the malleability of art that it must reduce it to a set of code. Nor do I believe that there can be no underlying set of factors that can help a novel become a hit.

But while generalisation is important to science, for art, it can be a straightjacket. A factory model of churning out variations of the same novel over and over again, serves no one. Yes, you might argue that some authors do this with great success, but for the vast majority of us, I would imagine our favourite novels are not ones that are merely an example from a handful of similar titles. At a guess, I would say our favourites are novels that stick out in our minds as unique. Ones that reverberate around our very souls and become important to our sense of identity.

There’s a great scene in the movie I-Robot where the robot (Sonny) tells Detective Spooner about his dreams.

Spooner: Robots don’t feel fear. They don’t feel anything. They don’t get hungry, they don’t sleep-

SonnyI do. I have even had dreams.

SpoonerHuman beings have dreams. Even dogs have dreams, but not you. You are just a machine; an imitation of life. Can a robot write a symphony? Can a robot turn a… canvas into a beautiful masterpiece?

Sonny[with genuine interest] Can you?

I think it’s a brilliant commentary because it speaks directly to the question of whether, art can be generalised and still move us, and, if it can, do we really need to be human in order to do it? I’m torn by these ideas. On the one hand I know that there are rules to the novel writing game. On the other, I find it difficult to believe that stories that make me think and feel all the feels can be reduced to a simple algorithm. Are we all that predictable? I really hope not.the circle

What I’m left with here is a question of what big data can teach us about the novel. On some level it shows there are elements to novels that, consciously or unconsciously, many people find satisfying and want more of. Yet, I can also see that, because human beings are not identical to each other, our reactions to any bestseller will still be unique – something that big data cannot predict. For example, the bestseller code concludes that the ‘perfect’ novel based on their findings is The Circle by Dave Eggers – a book which I enjoyed a bit but certainly did not consider ‘ideal’.

Anyway, what do you think? Will the future of the novel be books written by AI? Or maybe AI science will develop because of this new understanding of what makes people tick? Can a bestseller be reduced to a few simple determinants? And what does it say about those of us who are trying to write novels?