How do I know this? Listen to the authors talk to one another about the books. The majority of the time, what you hear them talking about is not the book as a complete package; instead, you hear them talking about a scene they wrote or a character they loved. In other words, they have spent a great deal of time crafting this smaller element, but forgotten the big picture of things.
I was looking at one of the recent editions of Romantic Times this morning and looking through the reviews and I saw again, another example of just this problem. Many of the stories that earned poor reviews had similar comments. The characters in this book were great but the plot (or insert any other single element) didn't work and brought the whole story down.
Now don't get me wrong. We love those individual scenes you have crafted. We love those secondary characters that really make a scene worthwhile, but, as a writer, it is crucial that you examine the whole package as well.