springload

(Originally published in ClickZ, February 2003) by Eric Picard

In Part 1, I discussed three major problems rooted in online ad technology’s architecture. Now, possible solutions.

Neither is final, but at least we have a starting point for discussion. For those of you who read this for the “sparkling wit” of my commentary, bear with me. This is serious stuff for serious times.

Any solution must be industry-wide. It isn’t something one company working alone can achieve. Many approaches are possible. Below, two possible directions, both of which address the following:

• Media costs are too low to support the industry. The widely accepted solution is conversion to reach-and-frequency-based media buying and selling. There’s no way to implement this today due to technical barriers.
• Discrepancies between site-side and third-party ad servers are too high. Both proposals remove serving duplication by the publisher and third-party ad servers. Only one counts impressions.
• Inventory control systems are too unpredictable. My proposals make this more predictable by streamlining inventory control system function and selling by audience as opposed to general volume.

Let me clarify my definitions of ad servers. I refer to an ad server representing a publisher’s inventory as a “site-side ad server” (SSAS). An ad server delivering an advertiser or agency campaign on its behalf is a “third-party ad server” (3PAS).

Some basic assumptions are made in each proposal (you may disagree):

• Reach and frequency planning and buying are desired to increase online media’s value.
  • Media planners/buyers need tools covering reach and frequency/gross ratings points.
  • Third-party ad serving is required for multiple-publisher buys.
• Business models are as important to the architectural changes as working technology.
• Adopting a new architecture will be a long process but must happen within two years. A plan must be achievable in that timeframe.

Architecture Today

Current architecture separates SSASs and 3PASs. Integration issues include:

• A 3PAS is managed by the site-side system through a redirect, but site-served ads are not. This is the root of the discrepancies between the systems.
• Advertisers pay twice to serve ads: to the publisher and the 3PAS company.
• There’s no mechanism for a 3PAS to request the SSAS change how the media is rotated or handled. This is required to support cross-site frequency capping (a root requirement of reach and frequency).
• Inventory control is viewed as a linked system to site-side serving. No mechanisms support views into inventory control from the 3PAS or from media planning tools.
• Little integration exists between all systems used in online advertising, and virtually no automation exists.

Cookies: A Major Barrier

A barrier to any workable system overhaul is HTTP cookie handling. Due to cookies’ use restrictions, a solution enabling identification of unique users between publisher and 3PAS is needed. Not a privacy issue — this is anonymous. To track and value activity of aggregate anonymous users, a mechanism is needed to recognize them when they intersect with a campaign across multiple publishers.

There are two schools of thought resolving this issue. One advocates a universal cookie carefully shared across the industry with companies agreeing to a strict privacy policy. The other calls for some kind ofDNS-alias-creating mechanism. Both are complex, but it’s believed resolution is possible. The solutions below assume the cookie problem is resolved.

Solution 1: Divorce Inventory Control From Ad Serving

Change the relationship between inventory-control and ad-serving systems. The 3PAS’s nature won’t change much. It still serves its own content and the site maintains control over inventory. But hierarchy changes. In essence, the site separates inventory control from content delivery. Any ad serve is treated as a redirect, whether served by the publisher or third party. The impression is tracked at the same point for either server.

Pros:

• Media pricing will be “purer.” Ad delivery cost is separate from media cost.
• The advertiser can have content delivered by the SSAS or 3PAS. Paying twice for serving isn’t an issue. This reduces costs.
• The ad server — site side or third party — remains on one level, eliminating the discrepancy issue.
• This supports the need for the 3PAS to serve its own content to track rich media activity (particularly Flash).
• It creates more room for innovation when a third party serves the content. 3PAS can differentiate technically from others with varied capabilities for rich media tracking or other solutions, such as automated optimization.
• Inventory control is simplified, with a more specific focus on that part of the system without concern for ad serving.

Cons:

• Third-party serving fees are not reduced. Savings would have to be on the media side, when the publisher didn’t serve the content.
• Publishers must trust the 3PAS for tracking or open a pretty deep API (which many are reluctant to do) for tracking integration.
• Advertisers must accept publisher media volume numbers (impression opportunities) as distinct from the impression number given by the ad server. A business model shift, but the fairest way for all parties to be paid an equitable fee. Billing media on page views versus ad serves is an old debate.

Walk-through:

1. The media planning tool hooks into the publisher’s trafficking and account management system and queries available inventory. It recommends inventory to the agency or advertiser based on criteria.
2. The media planning tool exports campaign information to the 3PAS.
3. The 3PAS traffics the campaign to the publisher’s trafficking and account management system (a component of the SSAS).
4. A user visits the publisher’s site through a browser. A call is made to the publisher’s server, generating an HTML page.
5. While generating the page, the publisher’s server calls the inventory control system for an ad tag. The system selects the relevant ad tag based on available inventory, confirms the user is not subject to frequency capping, and returns that ad tag to the publisher.
6. Depending on the architecture, a call to the ad server (SSAS or 3PAS) may occur. Most likely, this step is skipped.
7. The page is rendered in the user’s browser, which reads the ad tag. A call is made to the ad server (SSAS or 3PAS) for the creative. Methodology (including the redirect of the user) is identical for all ad servers.

Solution 2: Third-Party Servers Stop Serving Ads

Solution two removes ad serving from the 3PAS. It turns the 3PAS into a campaign management, data collection, and cross-publisher media management system. On the surface, publishers are comfortable. Their side looks much as it does now. But most advertisers will find limitations on rich media and reporting problematic.

Pros:

• Media pricing model stays much as it is. The publisher charges a mixed cost for ad delivery and inventory.
• Third-party ad-serving fees drop (the majority of the charge today is for ad delivery costs).

Cons:

• This solution requires more work and trust between publishers and 3PASs.
• Reporting timeframes are restricted to what the publisher can support. Most use log files, meaning data will not be updated within campaigns for longer periods of time than most 3PAS customers are used to.
• There is no way to use a 3PAS to “audit” publisher activity. The publisher supplies all impression and click data. With the broader mix of 3PASs on the market, advertisers and agencies can choose one they feel is accurate rather than rely on the accuracy of the publisher’s tracking.
• Many innovations are reduced in effectiveness or simply not possible, including rich media tracking (which requires a 3PAS to serve the creative). It relies on publisher-side solutions, traditionally less advanced than those offered by 3PASs and rich media companies.

Walk-through:

1. The media planning tool hooks into the publisher’s trafficking and account management system and queries available inventory. It recommends inventory to the agency or advertiser based on criteria.
2. The media planning tool exports campaign information to the 3PAS.
3. The 3PAS traffics the campaign to the publisher’s trafficking and account management system (a component of the SSAS).
4. A Web user visits the publisher’s site through a browser. A call is made to the publisher’s server, generating an HTML page.
5. While generating the page, the publisher’s server calls the inventory control system for an ad tag. The system chooses the relevant ad tag based on available inventory, confirms the user isn’t subject to frequency capping, and returns the relevant ad tag to the publisher.
6. Depending on architecture, a call to the 3PAS may occur. Most likely, this step is skipped.
7. The page is rendered in the user’s browser, which reads the ad tag. A call is made to the SSAS for the creative.
8. Impression and click data passed back to the 3PAS.

These solutions were assembled in my “spare” time. I’m not a programmer or an architect. I just happen to have a good brain for how technology works and an understanding of how things are configured today. My goal is to get things moving. To be a catalyst for change. Without someone taking a stab at this, I fear nothing will happen.

(Originally published in ClickZ, December 2002) by Eric Picard

A few months ago, I wrote an article about some of the problems with online advertising. As a follow up, today I’m going to discuss some major industry problems, but this time from the perspective of how these problems can be addressed. Those problems are:

• Media costs are too low.
• Discrepancies between site-side ad servers and third-party ad servers are too high.
• Inventory control systems are too unpredictable.

I’m going to be so bold as to offer a solution to the industry — probably not the actual solution, but apotential solution — that is, a viable solution that, at least on paper, would solve these problems plaguing the industry.

Today I’ll lay the groundwork for my proposal, and next month I’ll offer a version of the proposal to the industry. This is a big undertaking… and a bit of a risk. Frankly, I know the solution I come up with is not going to be a final solution, but at least it will be a starting point for discussion.

First, let me clarify my definitions of ad servers (for the millionth time) just to make sure I don’t get yelled at by misunderstanding vendors. I’ll refer to any ad server representing a publisher’s inventory as a “site-side ad server” (SSAS). Any ad server that delivers an advertiser or agency campaign on their behalf I’ll call a “third-party ad server” (3PAS).

How Do We Get Online Media to Be Valued Properly?

As I said last time, media costs are low for a number of reasons. But the general consensus in the industry (not complete consensus, mind you) is costs are low because “traditional” offline media teams cannot plan and buy media online the same way they do offline. There isn’t even a translation mechanism available, as far as I am concerned. Therefore, online media does not get valued properly.

Offline media teams plan their buys using reach- and frequency-based tools. The idea is simple. Reach is defined as the unique audience who saw an ad over the course of four weeks. Frequency is the number of times you reached the members of that audience over four weeks. This translates into a system of gross rating points (GRPs) by which each media vehicle is valued.

A number of companies are developing media planning solutions for the online space based on offline metrics. But even if they succeed, there are still big holes that must be filled.

Offline media is very predictable compared to online media. A radio media plan, for instance, is very accurate when it is bought — the planner knows almost exactly how many GRPs he is going to get. Online media is much less reliable — it fluctuates wildly. This means a stabilization factor must be added to online media buying to compensate for this.

Luckily, there is an answer — frequency caps. The idea is simple. Once an individual member of your target audience has seen your ad the desired number of times (across publishers), turn off your campaign for that user.

Unfortunately, implementation isn’t as simple as the idea, because currently no technology is availablethat the market could adopt to meet this need. It just isn’t available today, nor is it possible for one company alone to build this solution. A broad cross-industry solution is needed to enable this, and it won’t be easy to build.

Media Planning, Buying, Trafficking, and Discrepancy Resolution

The next two problems may seem very unrelated but are actually tied together in the chain of processes within our industry. Let’s look first at the way our industry processes work together.

As you can see in the graphic below, the business model for working online is complex.

For an ad to actually appear on a Web site:

1. The media planner must research the available Web sites and assemble a group of sites from which to buy inventory.
2. A media buyer contacts the sales teams at the short list of publishers she’s interested in with a request for proposals (RFP).
3. The publisher’s media sales team must review the company’s inventory management system to see if the inventory is available, generate an RFP, and send it to the media buyer.
4. The media buyer reviews the RFP and, if approved, sends an insertion order (IO) over to the media salesperson.
5. Next, the IO is handed off to the ad operations team on both ends.
6. The publisher ad operations team gets the space reserved within inventory control.
7. The agency (or advertiser, if it’s handled internally) creative team and ad operations team put the campaign into a 3PAS.
8. The creative team creates the ads; the placements are generated with creatives assigned.
9. The agency ad operations team traffics the ads through the 3PAS to the publisher.
10. The publisher ad operations team picks up the ad tags through the 3PAS and places the ad tags into the SSAS.
11. The campaign runs, and the agency reviews reports from both the 3PAS and the SSAS.

Now, in a perfect world, the campaign would run perfectly. The publisher would run the exact number of impressions specified in the agency’s IO. The impressions and clicks shown by the publisher’s SSAS and those shown by the agency’s 3PAS would match exactly. Unfortunately, in the real world, this just isn’t common.

Most likely, the campaign will be either under- or overdelivered by the SSAS. This happens for many reasons, but primarily because the problem of managing real-time inventory is very difficult. Also, the media buy is negotiated by volume of general delivery rather than by audience delivery, which makes the whole thing less predictable.

In addition, there will always be a discrepancy between the SSAS and the 3PAS. It is inherent in the nature of the technology — the SSAS counts before the 3PAS does, and users sometimes close a browser or click an available link before the ad call is received by the third party. This is just a basic fact. Because the SSAS and the 3PAS count separately, the likelihood of them having matching numbers is almost nonexistent.

These are three big problems. How do we fix them? I’ll tell you my answer next time. But I’ll leave you with one last issue to ponder.

Today, SSAS and 3PASs support the workflow model I’ve shown above. Below is a diagram of the way these systems interact, so ads run on the user’s browser when she views a Web page.

When a user calls a publisher’s Web site in her browser:

1. The publisher’s Web server asks the SSAS inventory control system for an ad tag.
2. The browser calls back to the SSAS for the ad (an impression is recorded) and is then redirected to the 3PAS.
3. The 3PAS delivers the ad to the browser (and counts an impression).
4. The user clicks on the ad, is sent to the SSAS (a click is counted) — and then is redirected to the 3PAS.
5. The 3PAS counts the click, then redirects the user to her final destination.

This process is far too complicated. Even though these redirects typically take less than a second, the fact we’re counting at different times makes discrepancies unavoidable. This model also requires publishers to bear the brunt of coordinating complex delivery schemas, when they’re already dealing with a difficult inventory control issue.

In addition, this system requires publishers to continually upgrade their ad-serving systems to manage increasingly complex rich media implementations. Meanwhile, the 3PAS has a far more demanding reporting role and must serve much of the rich media content anyway, so rich media and post-event (beyond banner) tracking can occur.

My proposal will be a comprehensive plan to solve all three of these problems in one rebuilding of the existing process. I hope this column and the next will spark some ideas with readers — ideas that will move the industry forward. See you next time when I make my proposal.

(Originally published in ClickZ, October 2002) by Eric Picard

What does this industry need to fix? I’ll delve into three specific problems here… for starters.

Media Costs Too Low

This might not seem like a problem. For many marketers, it’s a godsend. But it’s a huge snafu. Media costs are depressed, making publishers’ margins too low. The cost of building and maintaining their sites and of serving ads are too close to the amount of money they bring in.

Media sales commissions are hardly lucrative. Really great salespeople won’t stick around in an industry where they can’t make money, because great salespeople are driven by a desire for wealth.

From an operations standpoint, the only way to effectively implement and manage an online advertising campaign across a large number of publishers (from the marketer/agency perspective) is to use a third-party ad server (3PAS) to manage, traffic, and report on campaign activity (see my column on 3PAS). Two years ago, using a 3PAS was a no-brainer for anyone who understood the value proposition.

At that time, media costs averaged around $15-20 CPM. The 3PAS costs were around $1-3 CPM. This was 5 to 10 percent of the media buy. Today, media costs hover around $2 CPM, and 3PAS costs around $0.75 — 20-40 percent of the cost of the buy.

For marketers and agencies that understand the value proposition of a 3PAS, value still significantly outweighs cost. Without a 3PAS, it’s difficult or impossible to calculate return on investment (ROI) and a campaign’s true value. Unfortunately, some advertisers won’t OK the additional cost to agencies managing their online campaigns. The advertiser loses the ability to properly evaluate campaigns, as site-side servers don’t offer post-event tracking (post-impression and post-click analysis). The additional labor involved in trafficking and reporting without a 3PAS cancels out the savings.

Ad-Server Discrepancies Still Too High

A few years ago if you asked what the biggest industry problem was, there’s a good chance the response contained the word “discrepancy.” Nothing’s changed.

Example: Advertiser buys 1 million impressions from publisher. The advertiser traffics the ads using a 3PAS to the publisher. The campaign runs. When it’s over, the publisher’s report shows it served 1 million impressions. The 3PAS shows a lower number. In some cases, a significantly lower number.

The industry average discrepancy rate is 20 to 30 percent, depending on who you ask. A few people tell me they regularly experience 40 percent discrepancies. That’s a big difference. The problem gets nastier. Since the advertiser has an available lower number, it often tries to get the publisher to accept the 3PAS number and lower the bill accordingly. This causes the publisher to request a discrepancy investigation, which costs both the site-side server and the 3PAS time and money.

Why discrepancies? By definition, there should be discrepancies. Really. The publisher serves the ad tag that calls for the ad. At that moment, the publisher counts the impression. The call goes to the 3PAS, which serves the ad and counts the impression at that moment. While the call is being sent the user could close her browser or click away, interrupting the process before the 3PAS counts the impression. By nature, there should be some discrepancy. Worse, some scenarios have 3PAS with ahigher number of impressions than the publisher, counterintuitive as that seems.

One cause is from publishers who filter IP addresses of employees. This is prevalent at large networks that systematically filter out employees at their own IP or domain. As these networks have a large internal population visiting their properties, this alone can cause a significant discrepancy.

Possibly the most common cause of this type of discrepancy between site-side server and 3PAS is caching (a Web page is stored locally for quick future retrieval). If the publisher counts page views rather than ad calls (as IAB counting methodology guidelines require), the publisher may not be aware of a page request if the content is cached. As 3PAS generally “busts cache” effectively, the advertiser may have higher numbers than the publisher. Its number would be more accurate. Since the advertiser got a deal from the publisher in this case, it generally doesn’t complain about this discrepancy. Effectively, the publisher gives the advertiser free impressions.

What’s an acceptable discrepancy? Only you can answer that. Most experts say 10 to 15 percent is OK. If you’re getting a lower discrepancy, you should be happy. When evaluating ad servers, ask the following questions:

• What’s the average discrepancy between site-side servers and your product? If it’s over 15 percent, ask why.
• What’s your discrepancy resolution policy? This sets expectations for resolution timelines and what form resolution may take.
• What’s server response time (the time it takes the server to choose an ad and count the impression while sending the response back to the user)? The lower the response time (sometimes called matching speed), the lower the discrepancy. These should be measured in milliseconds. Due to the nature of Internet traffic, charting can show even a few milliseconds can cause significant changes in discrepancy rates.
• How do you bust cache? The technology should use at least two methods.
• Do you follow IAB counting methodology guidelines? This answer should be yes, and make use of the IAB robots and spiders list.

Technical Implications of IE 6 and Third-Party Cookies

Internet Explorer (IE) 6 has a default method for handling third-party cookies. IE 6 only allows cookies to be set by the domain visited, unless a third party has a P3P-compliant compact privacy statement.

This is relatively simple to set up. The third party confirms it’s not collecting personally identifiable user information. On paper, this sounds good. If you’re on Yahoo, for example, only Yahoo can set cookies on your PC unless the cookie is “safe,” meaning it won’t steal any personal information.

But Microsoft did this alone, and it doesn’t reflect the way the Internet works. It’s easy for ad servers. Most 3PAS and site-side servers (to my knowledge) are in compliance. Many publishers are owned by networks of sites that make use of consolidated cookies across all properties. In that case, if any of the servers that set cookies have different domain names, do collect personally identifiable information, or neglected to set up compact privacy statements properly, IE 6 blocks its cookies. This can cause problems: from the inconvenience of not having forms prepopulated or settings remembered to frequency caps not functioning on pop-ups (which really annoys users).

As you surf with IE 6, a red eye in the bottom of your browser’s window provides a privacy report when clicked. It’s illuminating to surf high-profile sites and learn they’re not P3P compliant.

There are plenty of other problems. These three are significant and often misunderstood. Let’s solve them and move forward!