It's not too difficult to explain anecdotal evidence of ESP, but scientific data is a little trickier. The common skeptical view is that parapsychologists' positive lab results come from problems with the experiment or plain old bad science. If the researcher's mathematical model is faulty, for example, simple chance guessing might look like something exceptional. Or the researchers could accidentally influence the subject to pick the right target. This could certainly happen, if the experiment isn't tightly controlled. In Rhine's early experiments, for example, he knew the correct card and he usually made eye contact with the guesser. The subject could have guessed correctly by unconsciously picking up on Rhine's body language -- what looked like ESP could have been simple intuition. Rhine later improved his methods, and his subject's accuracy did drop off somewhat.
The biggest problem with a lot of ESP research is it isn't reproducible. That is, one scientist may get results that another scientist can't get by replicating the experiment with different subjects. Parapsychologists might point out that some people aren't as psychically in-tune as others, so different subjects will yield different results, but the stigma still sticks. Reproducible results are essential to the conventional scientific method, so many scientists discount any irreproducible data no matter how credible the source.
In some cases, skeptics charge that ESP evidence is the product of outright fraud. There have certainly been experiments where parapsychologists manipulated data to support their own theories (this has occurred in most, if not all, scientific disciplines), and even an innocent scientist can have a hard time disproving these claims. Accusing a respected scientist of fraud is pretty serious business, however, so most skeptics are hesitant to take that step.
Skeptics are much more likely to point out fraud in unscientific ESP demonstrations, such as stage psychic shows. Most parapsychologists are also wary of ESP demonstrations for entertainment, simply because it's too easy to create the illusion of psychic powers.
The most ubiquitous fraudulent method is cold reading, in which the stage psychic rapidly throws out general, broad suggestions to an audience member until something "hits." For example, the psychic might say, "I see a man very close to you -- the name starts with J -- Jason, John, Jim? -- An uncle, or grandfather, or older friend?" Objectively, you can see that this is so broad it could apply in some way to just about everybody. But in the moment, subjects typically concentrate on any correct guesses and ignore everything that misses. It's the age-old trick of never being wrong and occasionally being right.