Category Archives: Ad Serving

Jul 11 2013
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Economics

How arbitrage works in digital advertising today

By Eric Picard (Originally published on iMediaConnection.com July 11, 2013)

The idea that ad networks, trading desks, and various technical “traders” in the ad ecosystem are “arbitraging” their customers is fairly well understood. The idea that an intermediary of some kind sells ad inventory to a media buyer, but then buys it somewhere else for a lower price is a pretty basic reality in our industry. But what most of us don’t understand is how it gets done and especially how technically advanced firms are doing it.

So let’s talk about this today — how arbitrage is enacted by various constituents, and I’d love to hear some reactions in the comments about how marketers and media buyers feel about it, especially if they weren’t aware of how it was done. Note: There are many ways to do this; I’m just going to give you some examples.

Old school networks

Back in the day, ad networks would go to large publishers and negotiate low price remnant buys (wholesale buys) where they’d buy raw impressions for pennies on the dollar, with the rule being that the network could only resell those impressions without identifying the publisher (blind inventory resale).

The networks that have done this well traditionally apply some targeting capabilities to sell based on context/content and also audience attributes. But even this is all very old school. The more advanced networks even back in the old days employed a variety of yield optimization technologies and techniques on top of targeting to ensure that they took as little risk on inventory as possible.

RTB procurement

Many networks now use the exchanges as their primary “procurement” mechanism for inventory. In this world there’s very little risk for networks, since they can set each individual campaign up in advance to procure inventory at lower prices than they’ve been sold. There is a bit of risk that they won’t be able to procure inventory — i.e. there isn’t enough to cover what they’ve pre-sold. But the risk of being left holding a large amount of inventory that went unsold is much lower and saves money.

Once you mitigate that primary risk and then add in the ability to ensure margin by setting margin limits, which any DSP can do “off the shelf,” the risk in managing an ad network is so low that almost anyone can do it — as long as you don’t care about maximizing your margins. That’s where a whole new class of arbitrage has entered the market.

Technical arbitrage

There are many different ways that companies are innovating around arbitrage, but I’ll give you the baseline summarization so you can understand why many of the networks (or networks that advertise as if they’re some kind of “services based DSP”) are able to be successful today.

Imagine a network that has built an internal ad platform that enables the following:

  • Build a giant (anonymous) cookie pool of all users on the internet.
  • Develop a statistical model for each user that monitors what sites the network has seen them on historically on a daily/day-of-week basis.
  • Develop a historical model showing how much inventory on each site tends to cost in order to win the inventory on the exchange, perhaps even each individual user.
  • When you run a campaign trying to reach a specific type of user, the system will match against each user, then in the milliseconds before the bid needs to be returned, the network’s systems will determine how likely they are to see this user that day — and if they will find them on sites where historically they’ve been able to buy inventory for less money than the one they’re on at the moment.
  • If the algorithm thinks it can find that user for less money, it will either bid low or it will ignore the bid opportunity until it sees that user later in the day — when it probably can win the bid.

This kind of technology is now running on a good number of networks, with many “variations” on this theme — some networks are using data related to time of day to make optimization decisions. One network told me that it finds that users are likely to click and convert first thing in the morning (before they start their busy day), in mid-morning surfing (after they’ve gotten some work done), after lunch (when they’re presumably trying to avoid nodding off), and in the late afternoon before going home for the day. They optimize their bidding strategy around these scenarios either by time of day or (in more sophisticated models) depending on the specific user’s historical behavior.

You shouldn’t begrudge the networks based too much on this “technical arbitrage,” since all that technology requires a significant upfront investment. They’re still giving you access to the same user pool — but one question that nags at me is that they may be giving you that user on sites that are not great.

It also begs the question that if these very technical networks are buying their inventory on a per-impression basis, all the stories about fraud get me a little worried. A truly sophisticated algorithm that matches a unique ID should be able to see that those IDs are getting too many impressions to be human. But I haven’t done any analysis on this — it’s just a latent concern I have.

Advertisement
Jul 08 2013
Leave a comment
3rd Party Data, Ad Ecosystem, Ad Serving, Ad Technology, Data Collection, Economics, Media Agencies, Predictions, Publishers, Targeting

Life after the death of 3rd Party Cookies

By Eric Picard (Originally published on AdExchanger.com July 8th, 2013)

In spite of plenty of criticism by the IAB and others in the industry, Mozilla is moving forward with its plan to block third-party cookies and to create a “Cookie Clearinghouse” to determine which cookies will be allowed and which will be blocked.  I’ve written many articles about the ethical issues involved in third-party tracking and targeting over the last few years, and one I wrote in March — “We Don’t Need No Stinkin’ Third-Party Cookies” — led to dozens of conversations on this topic with both business and technology people across the industry.

The basic tenor of those conversations was frustration. More interesting to me than the business discussions, which tended to be both inaccurate and hyperbolic, were my conversations with senior technical leaders within various DSPs, SSPs and exchanges. Those leaders’ reactions ranged from completely freaked out to subdued resignation. While it’s clear there are ways we can technically resolve the issues, the real question isn’t whether we can come up with a solution, but how difficult it will be (i.e. how many engineering hours will be required) to pull it off.

Is This The End Or The Beginning?

Ultimately, Mozilla will do whatever it wants to do. It’s completely within its rights to stop supporting third-party cookies, and while that decision may cause chaos for an ecosystem of ad-technology vendors, it’s completely Mozilla’s call. The company is taking a moral stance that’s, frankly, quite defensible. I’m actually surprised it’s taken Mozilla this long to do it, and I don’t expect it will take Microsoft very long to do the same. Google may well follow suit, as taking a similar stance would likely strengthen its own position.

To understand what life after third-party cookies might look like, companies first need to understand how technology vendors use these cookies to target consumers. Outside of technology teams, this understanding is surprisingly difficult to come by, so here’s what you need to know:

Every exchange, Demand-Side Platform, Supply-Side Platform and third-party data company has its own large “cookie store,” a database of every single unique user it encounters, identified by an anonymous cookie. If a DSP, for instance, wants to use information from a third-party data company, it needs to be able to accurately match that third-party cookie data with its own unique-user pool. So in order to identify users across various publishers, all the vendors in the ecosystem have connected with other vendors to synchronize their cookies.

With third-party cookies, they could do this rather simply. While the exact methodology varies by vendor, it essentially boils down to this:

  1. The exchange, DSP, SSP or ad server carves off a small number of impressions for each unique user for cookie synching. All of these systems can predict pretty accurately how many times a day they’ll see each user and on which sites, so they can easily determine which impressions are worth the least amount of money.
  2. When a unique ID shows up in one of these carved-off impressions, the vendor serves up a data-matching pixel for the third-party data company. The vendor places its unique ID for that user into the call to the data company. The data company looks up its own unique ID, which it then passes back to the vendor with the vendor’s unique ID.
  3. That creates a lookup table between the technology vendor and the data company so that when an impression happens, all the various systems are mapped together. In other words, when it encounters a unique ID for which it has a match, the vendor can pass the data company’s ID to the necessary systems in order to bid for an ad placement or make another ad decision.
  4. Because all the vendors have shared their unique IDs with each other and matched them together, this creates a seamless (while still, for all practical purposes, anonymous) map of each user online.

All of this depends on the basic third-party cookie infrastructure Mozilla is planning to block, which means that all of those data linkages will be broken for Mozilla users. Luckily, some alternatives are available.

Alternatives To Third-Party Cookies

1)  First-Party Cookies: First-party cookies also can be (and already are) used for tracking and ad targeting, and they can be synchronized across vendors on behalf of a publisher or advertiser. In my March article about third-party cookies, I discussed how this can be done using subdomains.

Since then, several technical people have told me they couldn’t use the same cross-vendor-lookup model, outlined above, with first-party cookies — but generally agreed that it could be done using subdomain mapping. Managing subdomains at the scale that would be needed, though, creates a new hurdle for the industry. To be clear, for this to work, every publisher would need to map a subdomain for every single vendor and data provider that touches inventory on its site.

So there are two main reasons that switching to first-party cookies is undesirable for the online-ad ecosystem:  first, the amount of work that would need to be done; second, the lack of a process in place to handle all of this in a scalable way.

Personally, I don’t see anything that can’t be solved here. Someone needs to offer the market a technology solution for scalable subdomain mapping, and all the vendors and data companies need to jump through the hoops. It won’t happen in a week, but it shouldn’t take a year. First-party cookie tracking (even with synchronization) is much more ethically defensible than third-party cookies because, with first-party cookies, direct relationships with publishers or advertisers drive the interaction. If the industry does switch to mostly first-party cookies, it will quickly drive publishers to adopt direct relationships with data companies, probably in the form of Data Management Platform relationships.

2) Relying On The Big Guns: Facebook, Google, Amazon and/or other large players will certainly figure out how to take advantage of this situation to provide value to advertisers.

Quite honestly, I think Facebook is in the best position to offer a solution to the marketplace, given that it has the most unique users and its users are generally active across devices. This is very valuable, and while it puts Facebook in a much stronger position than the rest of the market, I really do see Facebook as the best voice of truth for targeting. Despite some bad press and some minor incidents, Facebook appears to be very dedicated to protecting user privacy – and also is already highly scrutinized and policed.

A Facebook-controlled clearinghouse for data vendors could solve many problems across the board. I trust Facebook more than other potential solutions to build the right kind of privacy controls for ad targeting. And because people usually log into only their own Facebook account, this avoids the problems that has hounded cookie-based targeting related to people sharing devices, such as when a husband uses his wife’s computer one afternoon and suddenly her laptop thinks she’s a male fly-fishing enthusiast.

3) Digital Fingerprinting: Fingerprinting, of course, is as complex and as fraught with ethical issues as third-party cookies, but it has the advantage of being an alternative that many companies already are using today. Essentially, fingerprinting analyzes many different data points that are exposed by a unique session, using statistics to create a unique “fingerprint” of a device and its user.

This approach suffers from one of the same problems as cookies, the challenge of dealing with multiple consumers using the same device. But it’s not a bad solution. One advantage is that fingerprinting can take advantage of users with static IP addresses (or IP addresses that are not officially static but that rarely change).

Ultimately, though, this is a moot point because of…

4) IPV6: IPV6 is on the way. This will give every computer and every device a static permanent unique identifier, at which point IPV6 will replace not only cookies, but also fingerprinting and every other form of tracking identification. That said, we’re still a few years away from having enough IPV6 adoption to make this happen.

If Anyone From Mozilla Reads This Article

Rather than blocking third-party cookies completely, it would be fantastic if you could leave them active during each session and just blow them away at the end of each session. This would keep the market from building third-party profiles, but would keep some very convenient features intact. Some examples include frequency capping within a session, so that users don’t have to see the same ad 10 times; and conversion tracking for DR advertisers, given that DR advertisers (for a whole bunch of stupid reasons) typically only care about conversions that happen within an hour of a click. You already have Private Browsing technology; just apply that technology to third-party cookies.

May 30 2013
Leave a comment
Ad Serving, Ad Technology, Economics, Media Agencies, Online Media, Publishers

Which Type Of Fraud Have You Been Suckered Into?

By Eric Picard (Originally published by AdExchanger.com on May 30th, 2013)

For the last few years, Mike Shields over at Adweek has done a great job of calling out bad actors in our space.  He’s shined a great big spotlight on the shadowy underbelly of our industry – especially where ad networks and RTB intersect with ad spend.

Many kinds of fraud take place in digital advertising, but two major kinds are significantly affecting the online display space today. (To be clear, these same types of fraud also affect video, mobile and social. I’m just focusing on display because it attracts more spending and it’s considered more mainstream.) I’ll call these “page fraud” and “bot fraud.”

Page Fraud

This type of fraud is perpetrated by publishers who load many different ads onto one page.  Some of the ads are visible, others hidden.  Sometimes they’re even hidden in “layers,” so that many ads are buried on top of each other and only one is visible. Sometimes the ads are hidden within iframes that are set to 1×1 pixel size (so they’re not visible at all). Sometimes they’re simply rendered off the page in hidden frames or layers.

It’s possible that a publisher using an ad unit provided by an ad network could be unaware that the network is doing something unscrupulous – at least at first.  But they are like pizza shops that sell more pizzas than it’s possible to make with the flour they’ve purchased. They may be unaware of the exact nature of the bad behavior but must eventually realize that something funny is going on. In the same way, bad behavior is very clear to publishers who can compare the number of page views they’re getting with the number of ad impressions they’re selling.  So I don’t cut them any slack.

This page fraud, by the way, is not the same thing as “viewability,” which involves below-the-fold ads that never render visibly on the user’s page.  That fraudulent activity is perpetrated by the company that owns the web page on which the ads are supposed to be displayed.  They knowingly do so by either programming their web pages with these fraudulent techniques or using networks that sell fake ad impressions on their web pages.

There are many fraud-detection techniques you can employ to make sure that your campaign isn’t the victim of page fraud. And there are many companies – such as TrustMetrics, Double Verify and Integral Ad Science – that offer technologies and services to detect, stop and avoid this type of fraud. Foiling it requires page crawling as well as advanced statistical analysis.

Bot Fraud

This second type of fraud, which can be perpetrated by a publisher or a network, is a much nastier kind of fraud than page fraud. It requires real-time protection that should ultimately be built into every ad server in the market.

Bot fraud happens when a fraudster builds a software robot (or bot) – or uses an off-the-shelf bot – that mimics the behavior of a real user. Simple bots pretend to be a person but behave in a repetitive way that can be quickly identified as nonhuman; perhaps the bot doesn’t rotate its IP address often and creates either impressions or clicks faster than humanly possible. But the more sophisticated bots are very difficult to differentiate from humans.

Many of these bots are able to mimic human behavior because they’re backed by “botnets” that sit on thousands of computers across the world and take over legitimate users’ machines.  These “zombie” computers then bring up the fraudsters’ bot software behind the scenes on the user’s machine, creating fake ad impressions on a real human’s computer.  (For more information on botnets, read “A Botnet Primer for Advertisers.”) Another approach that some fraudsters take is to “farm out” the bot work to real humans, who typically sit in public cyber cafes in foreign countries and just visit web pages, refreshing and clicking on ads over and over again. These low-tech “botnets” are generally easy to detect because the traffic, while human and “real,” comes from a single IP address and usually from physical locations where the heavy traffic seems improbable – often China, Vietnam, other Asian countries or Eastern Europe.

Many companies have invested a lot of money to stay ahead of bot fraud. Google’s DoubleClick ad servers already do a good job of avoiding these types of bot fraud, as do Atlas and others.

Anecdotally, though, newer ad servers such as the various DSPs seem to be having trouble with this; I’ve heard examples through the grapevine on pretty much all of them, which has been a bit of a black eye for the RTB space. This kind of fraud has been around for a very long time and only gets more sophisticated; new bots are rolled out as quickly as new detection techniques are developed.

The industry should demand that their ad servers take on this problem of bot fraud detection, as it really can only be handled at scale by significant investment – and it should be built right into the core campaign infrastructure across the board. Much like the issues of “visible impressions” and verification that have gotten a lot of play in the industry press, bot fraud is core to the ad-serving infrastructure and requires a solution that uses ad-serving-based technology. The investment is marginal on top of the existing ad-serving investments that already have been made, and all of these features should be offered for free as part of the existing ad-server fees.

Complain to – or request bot-fraud-detection features from – your ad server, DSP, SSP and exchange to make sure they’re prioritizing feature development properly. If you don’t complain, they won’t prioritize this; instead, you’ll get less-critical new features first.

Why Is This Happening?

I’ve actually been asked this a lot, and the question seems to indicate a misunderstanding – as if it were some sort of weird “hacking” being done to punish the ad industry. The answer is much simpler:  money.  Publishers and ad networks make money by selling ads. If they don’t have much traffic, they don’t make much money. With all the demand flowing across networks and exchanges today, much of the traffic is delivered across far more and smaller sites than in the past. This opens up significant opportunities for unscrupulous fraudsters.

Page fraud is clearly aimed at benefiting the publisher but also benefitting the networks. Bot fraud is a little less clear – and I do believe that some publishers who aren’t aware of fraud are getting paid for bot-created ad impressions.  In these cases, the network that owns the impressions has configured the bots to drive up its revenues. But like I said above, publishers have to be almost incompetent not to notice the difference in the number of impressions delivered by a bot-fraud-committing ad network and the numbers provided by third parties like Alexa, Comscore, Nielsen, Compete, Hitwise, Quantcast, Google Analytics, Omniture and others.

Media buyers should be very skeptical when they see reports from ad networks or DSPs showing millions of impressions coming from sites that clearly aren’t likely to have millions of impressions to sell.  And if you’re buying campaigns with any amount of targeting – especially something that should significantly limit available inventory such as Geo or Income– or with frequency caps, you need to be extra skeptical when reviewing your reports, or use a service that does that analysis for you.

Mar 24 2013
Leave a comment
3rd Party Data, Ad Ecosystem, Ad Serving, Ad Technology, Data Collection, Online Media, Online Privacy, Targeting

We don’t need no stinkin’ 3rd party cookies!

By Eric Picard (Originally published on AdExchanger.com)

I’ve been writing about some of the ethical issues with “opt-out” third-party tracking for a long time. It’s a practice that makes me extremely uncomfortable, which is not where I started out. You can read my opus on this topic here.

In this article, I want to go into detail about why third-party cookies aren’t needed by the ecosystem, and why doing away with them as a default setting is both acceptable and not nearly as harmful as many are claiming.

 

First order of business: What is a third-party cookie?

When a user visits a web page, they load a variety of content. Some of this content comes from the domain they’re visiting. (For simplicity sake, let’s call it Publisher.com.) Some comes from third parties that are loading this content onto Publisher.com’s web site. (let’s call it ContentPartner.com.) An example would be that you could visit a site about cooking, and the Food Network could provide some pictures or recipes that the publisher embeds into the page. Those pictures and recipes sit on servers controlled by the content partner and point to that partner’s domain.

When content providers deliver content to a browser, they have the opportunity to set a cookie. When you’re visiting Publisher.com’s page, it can set a first-party cookie because you’re visiting its web domain. In our example above, ContentPartner.com is also delivering content to your browser from within Publisher.com’s page, so the kind of cookie it can deliver is a third-party cookie. There are many legitimate reasons why both parties would drop a cookie on your browser.

If this ended there, we probably wouldn’t have a problem. But this construct – allowing content from multiple domains to be mapped together into one web page, which was really a matter of convenience when the web first was created – is the same mechanism the ad industry uses to drop tracking pixels and ads onto publishers’ web pages.

For example, you might visit Publisher.com and see an ad delivered by AdServer.com. And on every page of that site, you might load tracking pixels delivered by TrackingVendor1.com, TrackingVendor2.com, etc. In this case, only Publisher.com can set a first-party cookie. All the other vendors are setting third-party cookies.

There are many uses for third-party cookies that most people would have no issue with, but some uses of third-party cookies have privacy advocates up in arms. I’ll wave an ugly stick at this issue and just summarize it by saying: Companies that have no direct relationship with the user are tracking that user’s behavior across the entire web, creating profiles on him or her, and profiting off of that user’s behavior without his or her permission.

This column isn’t about whether that issue is ethical or acceptable, because allowing third-party cookies to be active by default is done at the whim of the browser companies. I’ve predicted for about five years that the trend would head toward all browsers blocking them by default. So far Safari (Apple’s browser) doesn’t allow third-party cookies by default, and Mozilla’s Firefox has announced it will block them by default in the next version of Firefox.

Why I don’t think blocking third-party cookies is a problem

There are many scenarios where publishers legitimately need to deliver content from multiple domains. Sometimes several publishers are owned by one company, and they share central resources across those publishers, such as web analytics, ad serving, and content distribution networks (like Akamai). It has been standard practice in many of these cases for publishers to map their vendors against their domain, which by the way allows them to set first-party cookies as well.

How do they accomplish this? They set a ‘subdomain’ that is mapped to the third party’s servers. Here’s an example:

Publisher.com wants to use a web analytics provider but set cookies from its own domain. It creates a subdomain called WebAnalytics.Publisher.com using its Domain Name Server, or DNS. (I won’t get too technical, but DNS is the way that the Internet maps IP addresses – the numeric identifier of servers – to domain names.) It’s honestly as simple as one of the publisher’s IT people opening up a web page that manages their DNS, creating a subdomain name, and mapping it to a specific IP address. And that’s it.

This allows the third-party vendor to place first-party cookies onto the browser of the user visiting Publisher.com. This is a standard practice that is broadly used across the industry, and it’s critically important to the way that the Internet functions. There are many reasons vendors use subdomains, not just to set first-party cookies. For instance, this is standard practice in the web analytics space (except for Google Analytics) and for content delivery networks (CDNs).

So why doesn’t everybody just map subdomains and set first-party cookies?

First, let me say that while it is fairly easy to map a subdomain for the publisher’s IT department, it would be impractical for a demand-side platform (DSP) or other buy-side vendor to go out and have every existing publisher map a subdomain for them. For those focused on first-party data on the advertiser side, they’ll still have access to that data in this world. But for broader data sets, they’ll be picking up their targeting data via the exchange as pushed through by the publisher on the impression itself. For the data management platforms (DMPs), given that this is their primary business, it is a reasonable thing for them to map subdomains for each publisher and advertiser that they work with.

Also, the thing that vendors like about third-party cookies is that by default they work across domains. That means that data companies could set pixels on every publisher’s web site they could convince to place their pixels, and then automatically they would track one cookie across every site they visited. Switching to first-party cookies breaks that broad set of actions across multiple publishers into pockets of activity at the individual publisher level. There is no cheap, convenient way to map one user’s activity across multiple publishers. And only those companies that have a vested interest – the DMPs – will make that investment, and it will limit the number of small vendors who can’t make that investment from participating.

But, is that so bad?

So does moving to first-party cookies break the online advertising industry?

Nope. But it does complicate things. Let me tell you about a broadly used practice in our industry – one that every single data company uses on a regular basis. It’s a practice that gets very little attention today but is pretty ubiquitous. It’s called cookie mapping.

Here’s how it works: Let’s say one vendor has its unique anonymous cookies tracking user behavior and creating big profiles of activity, and it wants to use that data on a different vendor’s servers. In order for this to work, the two vendors need to map together their profiles, finding unique users (anonymously) who are the same user across multiple databases. How this is done is extremely technical, and I’m not going to mangle it by trying to simplify the process. But at a very high level, it’s something like this:

Media Buyer wants to use targeting data on an exchange using a DSP. The DSP enables the buyer to access data from multiple data vendors. The DSP has its own cookies that it sets (today these are third-party cookies) on users when it runs ads. The DSP and the data vendor work with a specialist vendor to map together the DSP’s cookies and the data vendor’s cookies. These cooking mapping specialists (Experian and Acxiom are examples, but others provide this service as well) use a complex set of mechanisms to map together overlapping cookies between the two vendors. They also have privacy auditing processes in place to ensure that this is done in an ethical and safe way to ensure that none of the vendors gets access to personally identifiable data.

Note that this same process is used between advertisers and publishers and their own DMPs so that first-party data from CRM and user registration databases can be mapped to behavioral and other kinds of data.

The trend for data companies in the last few years has been to move into DMP mode, working directly with the advertisers and publishers rather than trying to survive as third-party data providers. This move was intelligent – almost prescient of the change that is happening in the browser space right now.

My short take on this evolution

I feel that this change is both inevitable and positive. It puts more power back in the hands of publishers; it solidifies their value proposition as having a direct relationship with the consumer, and will drive a lot more investment in data management platforms and other big data by publishers. The last few years have seen a data asymmetry problem arise where the buyers had more data available to them than the publishers, and the publishers had no insight into the value of their own audience. They didn’t understand why the buyer was buying their audience. This will fall back into equilibrium in this new world.

Long tail publishers will need to rely on their existing vendors to ensure they can easily map a first-party cookie to a data pool – these solutions need to be baked by companies who cater to long tail publishers, such as ad networks. The networks will need to work with their own DMP and data solutions to ensure that they’re mapping domains together on behalf of their long tail publishers and pushing that targeting data with the inventory into the exchanges. The other option for longer tail publishers is to license their content to larger publishers who can aggregate this content into their sites. It will require some work, which also means ambiguity and stress. But certainly this is not insurmountable.

I also will say that first-party tracking is both ethical and justifiable. Third-party tracking without the user’s permission is ethically a challenging issue, and I’d argue that it’s not in the best interest of our industry to try and perpetuate – especially since there are viable and acceptable alternatives.

That doesn’t mean switching off of third-party cookies is free or easy. But in my opinion, it’s the right way to do this for long-term viability.

Mar 24 2013
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Online Media, Predictions, Publishers

What everyone should know about ad serving

By Eric Picard (Originally published in iMediaConnection.com)

Publisher-side ad servers such as DoubleClick for Publishers, Open AdStream, FreeWheel, and others are the most critical components of the ad industry. They’re responsible ultimately for coordination of all the revenue collected by the publisher, and they do an amazing amount of work.

Many people in the industry — especially on the business side of the industry — look at their ad server as mission critical, sort of in the way they look at the electricity provided by their power utility. Critical — but only in that it delivers ads. To ad operations or salespeople, the ad server is most often associated with how they use the user interface — really the workflow they interact with directly. But this is an oversight on their part.

The way that the ad server operates under the surface is actually something everyone in the industry should understand. Only by understanding some of the details of how these systems function can good business decisions be made.

Ad delivery

Ad servers by nature make use of several real-time systems, the most critical being ad delivery. But ad delivery is not a name that adequately describes what those systems do. An ad delivery system is really a decision engine. It reviews an ad impression in the exact moment that it is created (by a user visiting a page), reviews all the information about that impression, and makes the decision about which ad it should deliver. But the real question is this: How does that decision get made?

An impression could be thought of as a molecule made up of atoms. Each atom is an attribute that describes something about that impression. These atomic attributes can be simple media attributes, such as the page location that the ad is imbedded into, the category of content that the page sits within, or the dimensions of the creative. They can be audience attributes such as demographic information taken from the user’s registration data or a third-party data company. They can be complex audience segments provided by a DMP such as “soccer mom” — which is in itself almost a molecular object made up of the attributes of female, parent, children in sports — and of course various other demographic and psychographic atomic attributes.

When taken all together, these attributes define all the possible interpretations of that impression. The delivery engine now must decide (all within a few milliseconds) how to allocate that impression against available line items. This real-time inventory allocation issue is the most critical moment in the life of an impression. Most people in our industry have no understanding of what happens in that moment, which has led to many uninformed business, partnership, and vendor licensing decisions over the years, especially when it comes to operations, inventory management, and yield.

Real-time inventory allocation decides which line items will be matched against an impression. The way these decisions get made reflects the relative importance placed on them by the engineers who wrote the allocation rules. These, of course, are informed by business people who are responsible for yield and revenue, but the reality is that the tuning of allocation against a specific publisher’s needs is not possible in a large shared system. So the rules get tuned as best they can to match the overarching case that most customers face.

Inventory prediction

Well before the impression is generated and has to be allocated out to the impressions in real-time, inventory was sold in advance based on predictions of how much volume would exist in the future. We call these predicted impressions “avails” (for “available to sell”) in our industry, and they’re essentially the basis for how all guaranteed impressions are sold.

We’ll get back to the real-time allocation in a moment, but first let’s talk a bit about avails. The avails calculation done by another component of the ad server, responsible for inventory prediction, is one of the hardest computer science problems facing the industry today. Predicting how much inventory will exist is hard — and extremely complicated.

Imagine if you will that you’ve been asked to predict a different kind of problem than ad serving — perhaps traffic patterns on a state highway system. As you might imagine, predicting how many cars will be on the entire highway next month is probably not very hard to do with a pretty high degree of accuracy. There’s historical data going back years of time, month by month. So you could take a look at the month of April for the last five years, see if there’s any significant variance, and use a bit of somewhat sophisticated math to determine a confidence interval for how many cars will be on the highway in the month of April 2013.

But imagine that you now wanted to zoom into a specific location — let’s say the Golden Gate Bridge. And you wanted to break that prediction down further, let’s say Wednesday, April 3. And let’s say that we wanted to predict not only how many cars would be on the bridge that day, but how many cars with only one passenger. And further, we wanted to know how many of those cars were red and driven by women. And of those red, female-driven cars, how many of them are convertible sports cars? Between 2 and 3 p.m.

Even if you could get some kind of idea how many matches you’ve had in the past, predicting at this level of granularity is very hard. Never mind that there are many outside factors that could affect this; there are short-term issues that could help get more accurate as you get closer in time to the event such as weather and sporting events, and there are much more unpredictable events such as car accidents, earthquakes, etc.

This is essentially the same kind of prediction problem as the avails prediction problem that we face in the online advertising industry. Each time we layer on one bit of data (some defining attribute) onto our inventory definition, we make it harder and harder to predict with any accuracy how many of those impressions will exist. And because we’ve signed up for a guarantee that this inventory will exist, the engineers creating the algorithms that predict how much inventory will exist need to be very conservative on their estimates.

When an ad campaign is booked by an account manager at the publisher, they “pull avails” based on their read of the RFP and media plan and try to find matching inventory. These avails are then reserved in the system (the system puts a hold on avails that are being sent back to the buyer based for a period of time) until the insertion order (I/O) is signed by the buyer. At this moment, a preliminary allocation of predicted avails (impressions that don’t exist yet) is made by a reservation system, which divvies out the avails among the various I/Os. This is another kind of allocation that the ad server does in advance of the campaign actually running live, and it has as much (or even more) impact as the real-time allocation does on overall yield.

How real-time allocation decisions get made

Once a contract has been signed to guarantee that these impressions will in fact be delivered, it’s up to the delivery engine’s allocation system to decide which of the matching impressions to assign to which line items. The primary criteria used to make this decision is how far behind the matching line items are for successfully delivering against their contract, which we call “starvation” (i.e., is the line item starving to death or is it on track to fulfill its obligated impression volume?).

Because the engineers who wrote the avails prediction algorithms were conservative, the system generally has a lot of wiggle room when it comes to delivering against most line items that are not too complex. That means there are usually more impressions available when the impressions are allocated than were predicted ahead of time. So when all the matching line items are not starving, there are other decision criteria that can be used. The clearest one is yield, (i.e., of the available line items to allocate, which one of those lines will get me the most money for this impression?).

Implications of real-time allocation and inventory prediction

There’s a tendency in our industry to think about ad inventory as if it “exists” ahead of time, but as we’ve just seen, an impression is ephemeral. It exists only for a few milliseconds in the brain of a computer that decides what ad to send to the user’s machine. Generally there are many ways that each impression could be fulfilled, and the systems involved have to make millions or billions of decisions every hour.

We tend to think about inventory in terms of premium and remnant, or through a variety of lenses. But the reality is before the inventory is sold or unsold, premium or remnant, or anything else, it gets run through this initial mechanism. In many cases, inventory that is extremely valuable gets allocated to very low CPM impression opportunities or even to remnant because of factors having little to do with what that impression “is.”

There are many vendors in the space, but let’s chat for a moment about two groups of vendors: supply-side platforms (SSPs) and yield management companies.

Yield management firms focus on providing ways for publishers to increase yield on inventory (get more money from the same impressions), and most have different strategies. The two primary companies folks talk to me about these days are Yieldex and Maxifier. Yieldex focuses on the pre-allocation problem — the avails reservations done by account managers as well as the inventory prediction problem. Yieldex also provides a lot of analytics capabilities and is going to factor significantly in the programmatic premium space as well. Maxifier focuses on the real-time allocation problem and finds matches between avails that drive yield up, and it improves matches on other performance metrics like click-through and conversions, as well as any other KPI the publisher tracks, such as viewability or even engagement. Maxifier does this while ensuring that campaigns deliver, since premium campaigns are paid on delivery but measured in many cases on performance. The company is also going to figure heavily into the programmatic premium space, but in a totally different way than Yieldex. In other words, neither company really competes with each other.

Google’s recent release of its dynamic allocation features for the ad exchange (sort of the evolution of the Admeld technology) also plays heavily into real-time allocation and yield decisions. Specifically, the company can compare every impression’s yield opportunity between guaranteed (premium) line items and the response from the DoubleClick Exchange (AdX) to determine on a per-impression basis which will pay the publisher more money. This is very close to what Maxifier does, but Maxifier does this across all SSPs and exchanges involved in the process. Publishers I’ve talked to using all of these technologies have gushed to me about the improvements they’ve seen.

SSPs are another animal altogether. While the yield vendors above are focused on increasing the value of premium inventory and/or maximizing yield between premium and exchange inventory (I think of this as pushing information into the ad server to increase value), the SSPs are given remnant inventory to optimize for yield among all the various venues for clearing remnant inventory. By forcing competition among ad networks, exchanges, and other vehicles, they can drive the price up on remnant inventory.

How to apply this article to your business decisions

I’ve had dozens of conversations with publishers about yield, programmatic premium, SSPs, and other vendors. The most important takeaway I can leave you with is that you should think about premium yield optimization as a totally different track than discussions about remnant inventory.

When it comes to remnant inventory, whoever gets the first “look” at the inventory is likely to provide the highest increase in yield. So when testing remnant options, you have to ensure that you’re testing each one exactly the same way — never beneath each other. Most SSPs and exchanges ultimately provide the same exact demand through slightly different lenses. This means that barring some radical technical superiority — which none have shown me to be the case so far — the decision most likely will come down to ease of integration and ultimately customer service.

Nov 08 2012
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Economics, Media Agencies, Online Media, Publishers

How ad platforms work (and why you should care)

(By Eric Picard, Originally Published in iMediaConnection, 11/8/12)

Ad platforms are now open, meaning that startups and other technology companies can plug into them and take advantage of their feature sets. The ad technology space is now API driven, just like the rest of the web technology space. The significance of this change hasn’t hit a lot of people yet, but it will. The way this change will affect almost all the companies in ad technology will have an impact on everything: buying, selling, optimization, analytics, and investing.

Companies in our space used to have to build out the entire ad technology “stack” in order to build a business. That meant ad delivery (what most people think of as “ad serving”), event counting (impressions, clicks, conversions, and rich media actions), business intelligence, reporting, analytics, billing, etc. After building out all those capabilities, in a way that can scale significantly, each company would build its “differentiator” features. Many companies in the ad technology space have been created based on certain features of an ad platform. But because the ad platforms in our space were “closed,” each company had to build its own ad platform every time. This wasted a lot of time and money and — unbeknownst to investors — created a huge amount of risk.

Almost every startup in the ad platform space has existed at the whim of Google — specifically because of DoubleClick, the most ubiquitous ad platform in the market. When Google acquired DoubleClick, its platform was mostly closed (didn’t have extensive open APIs), and its engineering team subsequently went through a long cycle of re-architecture that essentially halted new feature development for several years. The market demanded new features — such as ad verification, brand safety, viewable impressions, real-time bidding, real-time selling, and others — that didn’t exist in DoubleClick’s platform or any others with traction in the space.

This led to the creation of many new companies in each space where new features were demanded. In some cases, Google bought leaders in those spaces. In others, Google has now started to roll out features that replicate the entirety of some companies’ product offerings. The Google stack is powerful and broad, and the many companies that have built point solutions based on specific features that were once lacking in Google’s platform suddenly are finding themselves competing with a giant who has a very advanced next-generation platform underlying it. Google has either completed or is in the process of integrating all of its acquisitions on top of this platform, and it has done a great job of opening up APIs that allow other companies to plug into the Google stack.

I’ve repeatedly said over the years that at the end of the natural process this industry is going through, we’ll end up with two to three major platforms (possibly four) driving the entire ecosystem, with a healthy ecosystem of other companies sitting on top of them. Right now, our ecosystem isn’t quite healthy — it’s complex and has vast numbers of redundancies. Many of those companies aren’t doing great and are likely to consolidate into the platform ecosystem in the next few years.

So how does the “stack” of the ad platform function? Which companies are likely to exist standalone on top of the stack? Which will get consumed by the stack? And which companies are going to find themselves in trouble?

Let’s take a look.

How ad platforms work (and why should you care)

Pretty much every system is going to have a stack that contains buckets of services and modules that contain something like what you see above. In an ideal platform, each individual service should be available to the external partner and should be consumable by itself. The idea here is that the platform should be decomposable such that the third party can use the whole stack or just the pieces it needs.

Whether we’re discussing the ubiquitous Google stack or those of competitors like AppNexus, the fact that these platforms are open means that, instead of building a replica of a stack like the one above, an ad-tech startup can now just build a new box that isn’t covered by the stack (or stacks) that it plugs into. Thus, those companies can significantly differentiate.

This does beg the question of whether a company can carve out a new business that won’t just be added as a feature set by the core ad platform (instantly creating a large well-funded competitor). To understand this, entrepreneurs and investors should review the offering carefully: How hard would it be to build the features in question? Is the question of growing the business one of technical invention requiring patents and significant intellectual property, or is it one of sales and marketing? Is the offering really a standalone business, or is it just a feature of an ad platform that one would expect to be there? And finally, will the core platforms be the acquirer of this startup or can a real differentiated business be created?

The next few years will be interesting. You can expect these two movements to occur simultaneously: Existing companies will consolidate into the platforms, and new companies will be created that take advantage of the new world — but in ways that require less capital and can fully focus on differentiation and the creation of real businesses of significance.

Oct 05 2012
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Economics, Online Media, Predictions, Startups

Entering the Fourth Wave of Ad Technology

By Eric Picard (Originally published on AdExchanger.com, 9/18/2012)

Ad tech is a fascinating and constantly evolving space.  We’ve seen several ‘waves’ of evolution in ad tech over the years, and I believe we’re just about to enter another.  The cycles of investment and innovation are clearly linked, and we can trace this all back to the late 90’s when the first companies entered the advertising technology space.

Wave 1

The early days were about the basics – we needed ways to function as a scalable industry, ways to reach users more effectively, systems to sell ads at scale, systems to buy ads at scale, analytics systems, targeting systems, and rich media advertising technology.

There was lots of investment and hard work in building out these 1.0 version systems in the space. Then the dot-com bubble imploded in 2001, and a lot of companies went out of business.  Investment in the core infrastructure ground to a halt for years. The price of inventory dropped so far and so fast that it took several years before investment in infrastructure could be justified.

We saw this wave last from 1996 through 2001 or 2002 – and during that dot-com meltdown, we saw massive consolidation of companies who were all competing for a piece of a pie that dramatically shrank.  But this consolidation was inevitable, since venture firms generally invest on a five to ten year cycle of return – meaning that they want companies to exit within an ideally 8 year window (or less).

Wave 2

The second wave was really about two things: Paid Search and what I think of as the “rise of the ad networks.”  Paid search is a phenomenon most of us understand pretty well, but the ad network phase of the market – really from 2001 to 2007 was really about arbitrage and remnant ad monetization.  Someone realized that since we had electronic access to all this inventory, we could create a ‘waterfall’ of inventory from the primary sales source to secondary sources, and eventually a ‘daisy-chain’ of sources that created massive new problems of its own.  But the genie was out of the bottle, and this massive supply of inventory that isn’t sold in any other industry was loosed.

It’s actually a little sad to me, because as an industry we flooded the market with super cheap remnant inventory that has caused many problems. But that massive over-supply of inventory did allow the third wave of ad tech innovation to get catalyzed.

Wave 3

Most people believe that the third wave was around ad exchanges, real-time buying and selling, data companies, and what I like to call programmatic buying and selling systems. But those were really just side effects. The third wave was really about building out the next generation infrastructure of advertising. Platforms like AppNexus and Right Media are not just exchanges; they’re fundamentally the next generation infrastructure for the industry.  Even the legacy infrastructure of the space got dramatic architectural overhauls in this period – culminated by the most critical system in our space (DoubleClick for Publishers) getting a massive Google-sponsored overhaul that among other thing opened up the system via extensive APIs so that other technology companies could plug in.

Across the board, this new infrastructure has allowed the myriad ad tech companies to have something to plug into.  This current world of data and real-time transactions is now pretty mature, and it’s extending across media types.  Significant financial investments have been made in the third wave – and most of the money spent in the space has been used to duplicate functionality – rather than innovate significantly on top of what’s been built.  Some call these “Me too” investments in companies that are following earlier startups and refining the model recursively.  That makes a lot of sense, because generally it’s the first group of companies and the third group of companies in a ‘wave’ that get traction. But it leads to a lot of redundancy in the market that is bound to be corrected.

This wave lasted from about 2005 to 2011, when new investments began to be centered on the precepts that happened in Wave 3 – which really was a transition toward ad exchanges (then RTB) and big data.

That’s the same pattern we’ve seen over and over, so I’m confident of where the industry stands today and that we’re starting to enter a new phase. This third major ad tech wave was faster than the first, but a lot of that’s because the pace of technology adoption has sped up significantly with web services and APIs becoming a standard way of operating.

Wave 4

This new wave of innovation we’re entering is really about taking advantage of the changes that have now propagated across the industry. For the first time you can build an ad tech company without having to create every component in the ‘stack’ yourself. Startups can make use of all the other systems out there, access them via APIs, truly execute in the cloud, and build a real company without massive  infrastructure costs.  That’s an amazing thing to participate in, and it wasn’t feasible even 3 years ago.

So we’ll continue to see more of what’s happened in the third wave – with infrastructure investments for those companies that got traction, but that’s really just a continuation of those third wave tech investments, which go through a defined lifecycle of seed, early, then growth stage investments.  Increasingly we’ll see new tech companies sit across the now established wave 3 infrastructure and really take advantage of it.

Another part of what happened in Wave 3 was beyond infrastructure – it involved the scaled investment in big data.  There have been massive investments in big data, which will continue as those investments move into the growth phase. But what’s then needed is companies that focus on what to do with all that data – how to leverage the results that the data miners have exposed.

Wave 4 will really change the economics of advertising significantly – it won’t just be about increasing yield on remnant from $0.20 to $0.50. We’ll see new ad formats that take advantage of multi-modal use (cross device, cross scenario, dynamic creatives that inject richer experiences as well as information), and we’ll see new definitions of ad inventory, including new ad products, packages and bundles.

So I see the next five years as a period where a new herd of ad tech companies will step in and supercharge the space. All this infrastructure investment has been necessary, because the original ad tech platforms were built the wrong way to take advantage of modern programming methodologies.  Now with modern platforms available pretty ubiquitously, we can start focusing on how to change the economics by taking advantage of that investment.

I also think we’re going to see massive consolidation of the third wave companies. Most of the redundancies in the market will be cleaned up.  Today we have many competitors fighting over pieces of the space that can’t support the number of companies in competition – and this is clearly obvious to anyone studying the Lumascape charts.

Unfortunately some of the earlier players who now have customer traction are finding that their technology investments are functionally flawed – they were too early and built out architectures that don’t take advantage of the newer ways of developing software. So we’ll see some of these early players with revenue acquiring smaller newer players to take advantage of their newer more efficient and effective architectures.

Companies doing due diligence on acquisitions need to be really aware of this – that buying the leader in a specific space that’s been around since 2008 may mean that to really grow that business they’ll need to buy a smaller competitor too – and transition customers to the newer platform.

For the investment community it’s also very important to understand that while Wave 3 companies that survive the oncoming consolidation will be very big companies with very high revenues, it is by nature that these infrastructure heavy investments will have lower margins and high volume (low) pricing to hit their high revenues. They still will operate on technology/software revenue margins – over 80% gross margins are the standard that tech companies run after. But the Wave 3 companies have seen their gross revenue numbers be a bit lower than we’d like as an industry.  This is because they are the equivalent of (very technically complex) plumbing for the industry.  There are plenty of places where they invest in intelligence, but the vast majority of their costs and their value deal with massive scale that they can handle, while being open to all the players in the ecosystem to plug in and add value.

Being a Wave 4 company implicitly means that you are able to leverage the existing sunk cost of these companies’ investment.  Thomas Friedman talks about this in “The World is Flat” – one of his core concepts is that every time an industry has seen (what he called) over-investment in enabling infrastructure, a massive economic benefit followed that had broad repercussions.  He cites the example of railroad investment that enabled cheap travel and shipping that led to a massive explosion of growth in the United States.  He cites the investment in data infrastructure globally that led to outsourcing of services to India and other third world countries on a massive scale.  And frequently those leveraging the sunk cost of these infrastructure plays make much more money from their own investments than those who created the opportunity.

So what should investors be watching for as we enter this fourth wave of ad tech innovation?

  1. Companies that are built on light cloud-based architectures that can easily and quickly plug into many other systems, and that don’t need to invest in large infrastructure to grow
  2. Companies that take advantage of the significant investments in big data, but in ways that synthesize or add value to the big data analysis with their own algorithms and optimizations
  3. Companies that can focus the majority of technical resources on innovative and disruptive new technologies – especially those that either synthesize data, optimize the actions of other systems, or fundamentally change the way that money is made in the advertising ecosystem
  4. Companies that are able to achieve scale quickly because they can leverage the existing large open architectures of other systems from Wave 3, but that are fundamentally doing something different than the Wave 3 companies
  5. Companies that take advantage of multiple ecosystems or marketplaces (effectively) are both risky but will have extremely high rewards when they take off

This is an exciting time to be in this space – and I predict that we’ll see significant growth in revenue and capabilities as Wave 4 gets off the ground that vastly eclipse what we’ve seen in any of the other waves.

Mar 03 2012
Leave a comment
Ad Serving, Ad Technology

Ad Serving 101 (Revised)

(Originally published in ClickZ, April 2007) by Eric Picard

Way back in October 2001, I wrote a column with this same title. To this day, I get numerous e-mail from people thanking me for covering this topic. Given the state of the market right now, it’s probably time for an update.

Ad serving is increasingly becoming a commodity. The actual delivery of ads is certainly already commodity. The term “ad serving” is misleading and misunderstood. It sounds like just something that coordinates an ad’s delivery. There’s much more going on here than just that. Lets walk through it.

Publisher Ad Serving

Let’s begin with the nuts and bolts, the most basic functionality of ad serving, then I’ll dive in and explain where the complexities lie. Below is the simplest scenario. An advertiser bought advertising from a publisher and sent the files to the publisher to be delivered onto the page.

Examples of Publisher Ad Servers include Doubleclick DART for Publishers (DFP), Accipiter Ad Manager, and 24/7 RealMedia’s Open Ad Stream (OAS).

Publisher Only Scenario:

Publisher_Ad_Server_1.jpg

  1. Browser points to a Web publisher and communicates via a publisher Web server. The publisher Web server responds back to the browser with an HTML file.
  2. In the HTML file is a pointer back to the publisher ad server. The browser calls the ad server looking for an ad. The ad server responds with the ad’s file location. In this case, the file is sitting on a content delivery network (CDN) such as Akamai or Mirror Image.
  3. The browser calls out to the CDN requesting the specific file containing the ad’s creative content (JPG, GIF, Flash, etc.). The CDN sends the file back to the browser.

This is relatively simple and easily understandable. But this deceptively simple diagram masks what’s going on behind the scenes at step 2. Let’s talk about that for a moment.

ad_server_backend.jpg

Every time an ad’s called, a series of very fast decisions and actions must take place. All this very detailed work should take only a few milliseconds:

  1. The ad delivery engine is called and handed an ID that’s unique to a specific Web page or group of Web pages.
  2. The delivery engine reads the ID and asks a sub-routine to choose which ad to delivery based on a bunch of facts – we’ll call this subroutine the “ad picker.”
  3. The ad picker has a very complex job. It must hold all sorts of data ready, typically in memory or in very fast databases.
    • Picker looks to see if the browser in question is part of any targeting groups in high demand, e.g. geographic location, gender or demographic data, behavioral groupings, etc.
    • Picker looks at all business rules associated with each campaign assigned to the unique identifier.
    • Picker looks at yield across the various options for each creative that match delivery criteria (which ad is most valuable to deliver at that moment).
    • Picker sends final ad selection to the delivery engine. The ad is sent to the browser.
  4. Data handed to inventory prediction system to help determine future ad availability and yield optimization.
  5. Delivery and performance data handed to the reporting and billing systems.

I’ve masked some of the incredible technical complexity, particularly around inventory prediction and yield optimization, but the moving parts are relatively easy to track. I haven’t discussed the business management features of the publisher systems. Bear in mind there are sales interfaces, order input interfaces, billing and reporting interfaces, and many other features I do a bit of disservice to in skipping over.

So that’s publisher-side ad serving, and it’s relatively straightforward. Let’s look at the advertiser side of the equation.

Advertiser/Agency Ad Serving

It’s a bit misleading to call advertiser/agency campaign management systems “ad servers.” These solutions do serve ads, but only as a function of tracking them. Rhere are technical realities in the market that require the serving of ads in order to track delivery across multiple publishers from a central source.

Why does an advertiser or agency use these tools? Two reasons: workflow automation and centralized reporting. These agency tools allow a big chunk of an agency’s grunt work to be automated; the data input, creative management and trafficking steps are significantly automated. Since these tools deliver the ads across all Web sites in a campaign, they centralize reporting into one report set comparatively showing all publishers.

Examples of Advertiser Side Ad Servers include the Atlas Suite, Doubleclick’s DART for Advertisers (DFA), Mediaplex’s Mojo, or Bluestreak’s IonAd system.

Agency Ad Serving Scenario:

advertiser_ad_server.jpg

  1. This begins as before: Browser points at a Web publisher, and communicates with a publisher Web server. The publisher Web server responds back to the browser with an HTML file.
  2. In the HTML file is a pointer back to the publisher ad server. The browser calls to the ad server looking for an ad. This is where it changes. Instead of the publisher ad server pointing toward its own CDN, the ad server delivers a secondary ad tag, a simple piece of HTML that points toward the agency ad server.
  3. The browser calls the agency ad server, which returns the final location of the creative in its own CDN.
  4. The browser calls to the agency ad server CDN requesting the specific file with the ad’s creative content (JPG, GIF, Flash, etc.). The CDN sends the file back to the browser.

While there’s an additional set of hops between browser and ad server in this scenario, bear in mind that this entire transaction takes less than a second. As before, this relatively simple set of actions makes the complexity of what’s happening seem much simpler.

Behind the scenes is a complex, business-facing workflow system that automates about half of the tasks in a media buyer or agency ad operations person’s job. Without this automation, already complex agency roles would be unbearably difficult.

The next step is to get the last half of the agency workflow mapped into these systems and really automate the tasks.

Mar 03 2012
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Rich Media

Technology Partners, not Vendors

(Originally published in ClickZ, December 2001) by Eric Picard

The online advertising industry requires technology to exist. More than any other ad medium, we’re technology-driven.

Recently, my colleague Tig Tillinghast wrote a mind-expanding article about the inefficiencies of the Web as an advertising medium. In it he states: “Newspaper can print a whole 32-page tabloid with 250 ads in it for $0.20 each, but it costs a Web site 25 percent more to serve the same number of ads.”

This may be true, but let’s analyze why it costs more. We’ve been printing (and advertising in) newspapers for literally hundreds of years. That’s plenty of time to figure out what the economies of scale are and to enable printing 250 ads for $0.20 each. The technology is relatively simple. Any changes over the past 50 years have only made it cheaper to produce a higher quality product.

You can’t compare the delivery of 250 online ads with that of 250 print ads, any more than you can compare print with TV ads. It’s apples to rhubarb.

The real world of online advertising requires certain technologies. Therefore, you need to forge relationships with your technology providers.

There’s a difference between viewing a technology provider as a vendor (weak relationship) or a partner (strong relationship). Dictionary.com defines “vendor” as: “One that sells or vends: a street vendor; a vendor of software products on the Web.” “Partner” is defined as: “One that is united or associated with another or others in an activity or a sphere of common interest.”

This should apply to your approach to tech providers. You can relate to them as a utility, like the electric or gas company. You can view them as a mission-critical service provider, like the phone company. Neither model suffices. You both have the same interests. They succeed if you succeed. Grow your business, you’ll have more need for their services.

Even services such as ad serving, often referred to lately as a commodity, are not mature enough to treat dismissively. There are not enough providers to choose from (and fewer after this year). There’s too much volatility, too much change, and too many issues for you to make that mistake.

Here are some basics about partnering with technology providers:

  • Yes, the economy sucks. No, you can’t beat up your tech partner on pricing to the point that it can’t turn a profit. It’ll go out of business, and that doesn’t help anyone. 
  • Find ways to pay your technology partner more by having it help grow your business. Tie your businesses together to gain the most from your partnership. Don’t slip into a valueless “paper partnership.” Apply resources and watch the relationship flourish. 
  • No company has every feature or widget you need. Product development (or customization) is required before you’ll be happy. A technology company will meet the needs of partners before worrying about customers who treat it like a vendor. 
  • Partners often participate on advisory councils and are involved in product research and development. This leads to measurable benefits, such as accessing new features before anyone else. You’re not likely to get that level of service from a mere vendor. 
  • Choose technology providers with upside potential and whose values match your company’s. Ask hard questions about service expectations, commitments to quality, and what the plan is when problems do arise (they always do).

Work from a common set of realistic expectations. There’s been so much over-promising and underdelivering in our industry that most customers are wary of committing to deeper relationships.

If a tech provider makes claims that set off your “spidey sense,” have it explain exactly how it’s going to live up to the claims. If high expectations are set by sales or marketing, ask to speak with a senior product or engineering rep to ensure the entire company is in alignment.

Make sure the company knows you want to partner with it. Be clear about expectations you have from a partnership as opposed to weaker relationships it might have with other customers. Make the opportunity attractive. Sell the company on your value as a partner. Expect and demand more. Be willing to cooperate. You’ll both come out ahead.

I hope your holidays were wonderful, and you and your family are approaching the new year with great expectations. –Eric

Mar 03 2012
Leave a comment
Ad Ecosystem, Ad Serving, Ad Technology, Media Agencies, Publishers, Startups

Advanced Ad-Serving Features, Part 2: Third-Party Ad Servers

(Originally published in ClickZ, November 2001) by Eric Picard

Last time, we discussed advanced features of site-side servers. Now let’s go deeper. This week, we’ll go into the even-more-advanced advanced features of third-party ad servers.

Third-party servers primarily serve the needs of advertisers and agencies. Sometimes they are called buy-side servers. They are part of the business infrastructure of these groups and must reliably and accurately deliver and report on ad serving and related user actions associated with the ads.

In addition to delivery and basic reporting, third-party servers provide unified comparative reporting for all publishers in a media buy, as well as many advanced features. From a feature standpoint, a third-party server is more complex than its site-side counterpart.

One thing to keep in mind: A third-party server is not able to “refuse” a call for an ad. If an ad tag is supplied from a third-party server to a site-side server and that ad is called, it must be served. Only a site-side server can schedule and deliver ad calls to users.

Beyond Banner Tracking

This is the big feature. Tracking beyond the banner enables the view of an ad session from impression to conversion (and beyond). This is a major reason a third-party server is a must for most advertisers. Some tracking types beyond the banner are:

  • Tracer tags. Tracer tags are single-pixel images placed on pages of the advertiser’s Web site so that activity on those pages can be correlated to the view or click of an ad. 
  • Post-click analysis. The user sees an ad and clicks on it. She arrives at a landing page on the advertiser’s Web site. She travels across three pages that have tracer tags on them. Each intersection of creative/tracer is credited to the advertiser’s reports. 
  • Post-impression (also called post-view) analysis. The user sees an ad but doesn’t click on it. That user (remembering the message) later travels to the advertiser’s Web site on his own. He moves across a number of pages with tracers on them. Each intersection of ad and tracer is correlated and credited to the advertiser’s reports. This analysis is a definitive branding measurement and is sometimes called a brand response report. Not all third-party servers collect post-impression data.

Reporting

  • Cross-publisher reports. A major reason to use a third-party server is that reports are covered across all publishers within a campaign. 
  • Comprehensive data sets. Since both post-view and post-click data must be recounted, reports must be unified and comprehensive.

Analytics

Some third-party servers offer advanced analytics capabilities. This is one of the fastest growing areas in the industry. Far more data is captured in an online ad campaign than in an offline one. Turning that data into actionable information isn’t simple. It takes days or weeks of human intervention and interpretation.

A powerful analytics package solves these problems by providing tools to get at actionable information more quickly. There are two basic types of tools to discuss:

  • Online analytical processing (OLAP) tool. This very powerful analytics tool enables the most control of data and reporting. Great power and flexibility comes at a great price, and few people are technical enough to use an OLAP tool to manipulate their data. In most agencies there are only a few, if any, people who can use these tools. It gets even sparser at the advertiser level. 
  • Wizard. To address problems with OLAP, some companies have started coming up with wizard-based interfaces for the most commonly asked questions. A good wizard-based interface can likely answer such questions as: Which publisher is the best media buy for my campaign goals based on the past six months of running ads across various publishers?

Optimization

Analytics deals with historical analysis to improve ongoing and future campaigns. Optimization deals with live campaigns that must be improved while still running. When done by hand (as is most often the case), only so much can be changed. Humans can optimize to a level of detail only so deep. This is best handled by technology, which provides much deeper analysis of data. Two types of optimization are:

  • Real time. Real-time optimization is the most powerful. Changes are made automatically to creative in rotation across placements based upon actual results read by the optimization tool. Real-time optimization requires real-time data to make changes. Few ad servers use a real-time reporting architecture, relying instead on 24-to-48-hour-delayed data. Real-time benefits include microtrend discovery (intraday changes in behavior within placements) and greater lift based on feedback loops. Additionally — if the system doesn’t make changes automatically, relying instead upon human approval or intervention — the lift is going to be lower. 
  • Recommendation. For situations where real-time data isn’t available, recommendation-based systems are the alternative. These systems read data when available and provide a list of recommendations to enable the customer to make changes. This inherently is a poorer performing model as changes are not happening quickly. Therefore, additional learning for the optimization tool is lost. The faster changes are made, the better the system gets at predicting performance. Still, this is a better method than hand optimization.

Targeting

  • Geographic targeting. Geotargeting is similar to site-side servers but somewhat less effective. You pay for the media regardless of whether you had an appropriate creative for the users an ad was served to. Wherever possible, try to geotarget at the publisher level. 
  • Profile-based targeting. As I detailed last time, ads can be targeted based on Web-surfing habits. Third-party ad servers have the same issues as site-side servers do. 
  • Session-specific targeting. Specifics include domain, browser type, and operating system. Again, this can be accomplished on the site side, usually to greater effect as the publisher only shows the ad (and bills you) when there is an appropriate fit. When served by a third party, you pay for the media even if it doesn’t fit your demographic.(Remember, there are plenty of other types of targeting I’m not covering here).

Trafficking Controls

Without a third-party server, trafficking ads to multiple publishers is a problem. It can be complex, with many points of failure. A good third-party server simplifies the process of trafficking campaigns and should provide valuable accounting methods for successful delivery and approval of your ads by the publisher.

Dynamic Ad Serving

Most publishers have a limit on the number of ads they will accept at one time. Usually this ranges from 5 to 10 creatives per week. Third-party servers use dynamic ad serving to rotate multiple creatives through one ad tag. This allows the advertiser/agency to traffic as many creatives associated with those tags as they want. This simplifies life for the advertiser and the publisher by cutting down significantly on the work done by both.

Conclusion

There are other ad server features not covered here. But this is a column, not a book! You should now be educated enough to talk to a salesperson without too much trepidation.

Next, I’ll write about a topic near and dear to my heart: how to work with tech companies for long-term success. It’s time to set a few things straight about this marketplace. Customers need to understand that while they are in a position to beat up their tech partners (notice I don’t call them vendors) on issues such as price, they should think twice. If there are any tech firms out there that would like to voice their thoughts on the topic, drop me a line.

← Older posts