VentureBeat, along with everybody else, covers a press release about the Pubmatic AdPriceIndex that I love, because it is data-driven, and will now proceed to pick apart because there is not enough data. In summary, this high level analysis of the data doesn’t really tell us enough about what is going on. Pubmatic should let us at the raw data so people can do a more nuanced analysis. Far be it from me to criticize people much, much, much smarter than me (PhDs!), but I think there are probably some aspects of the industry/Pubmatic offering that might not be understood and prevent the data from being controlled in a way that allows for effective analysis of the data set.
Anyway, I know I said in a previous post that I would try to be less harsh to people, but let’s dive in.
The most obvious question is with regard to the most prominent aspect of the press release. They announce that prices went down by a penny month over month. Of course, the first drilled down detail they offer is that small sites (under 1 million impressions) went down $0.32. They don’t provide compositional data, but it would stand to reason, given a data point like this, that small sites may have made up all of the penny shortfall. In fact, they do say that medium and large sites saw their CPM rise.
In fact, as their report spells out, big sites and medium sites have made higher CPMs every month! Maybe the recession is confined to small businesses? Not a chance.
So if we are saying that the penny shortfall was caused entirely by small web sites, the next question is how can we break down those sites. We have almost no insight into this, but one thing we can intuit is that they are working with many more small web sites this month than they did the previous month. If you read the June report, then you see that they worked with 3,500 web sites in that report, whereas they worked with 4,000 for the July report.
I bet most of those are small sites.
So they added a ton of new publishers to their service and prices fell. Web sites that weren’t being monetized previously or were being monetized poorly (either way, probably signs that it isn’t the most valuable inventory) signed up for Pubmatic, dragging price points down. That is not an unfair hypothesis.
One thing you could do is look at how publishers that participated in June saw their prices change month over month.
Alternately, the study indicates that as sites give Pubmatic more inventory, CPMs decline. Maybe they are signing up larger small sites (more profitable business for Pubmatic) and they are yielding lower new CPMs. Some analysis of how CPMs are linked to volume in Pubmatic’s system could explain this.
Considering even more alternatives, every ad network (and Pubmatic’s service) have testing costs. Maybe with the rapid influx of publishers and Pubmatic’s distribution of that publisher inventory over many advertising networks, the yield is artificially lowered by the need to test many new publishers across many new networks. Once again, controlling for the introduction of new publishers is important.
I am tired of typing now, but you can imagine the problems that might lurk in the other parts of the data. Incidentally, I bet that Pubmatic prints these great bio’s of the statisticians to imply that this is good, but I bet that at the very least the Chicago guy is embarrassed to be repping this data:
Albert Madansky, Ph.D. is the H.G.B. Alexander Professor Emeritus of Business Administration at the University of Chicago Graduate School of Business, and was the recipient of the 2005 American Statistical Association Founders Award.
Michele Madansky, Ph.D. is a media and market research consultant and former VP of Global Market Research for Yahoo!
Obviously, I work at an ad network, so don’t construe this as validation or not validation of what they are saying at all (prices rising, prices falling, don’t know and if I did I wouldn’t tell you). It is neither. This is simply a constructive critique of the information they reveal.