AU2013203585A1 - Content item pricing - Google Patents

Abstract

A threshold measure is determined for an advertisement based on one or more parameters associated with the advertisement. A determination is made as to whether the threshold measure exceeds a filter threshold. The advertisement is promoted if the threshold measure exceeds the filter threshold, and an actual cost per-click for the advertisement based on the one or more parameters and the filter threshold if the advertisement is promoted.

Description

AUSTRALIA Regulation 3.2 Patents Act 1990 Complete Specification Standard Patent Divisional APPLICANT: Google Inc. Invention Title: CONTENT ITEM PRICING The following statement is a full description of this invention, including the best method of performing it known to me: WO 2009/061535 PCT/US2008/072665 CONTENT ITEM PRICING CROSS-REFERENCE TO RELATED APPLICATION [0001] This application claims priority to U.S. Provisional Patent Application Ser. No. 60/954,722 filed August 8, 2007, and entitled "CONTENT 5 ITEM PRICING," and U.S. Utility Patent Application Ser. No. 11/836,960 filed August 10, 2007 entitled Content Item Pricing, the contents of which are incorporated herein by reference. BACKGROUND [0002] This disclosure relates to information retrieval. 10 [0003] Content items, e.g., advertisements, can be identified by a search engine in response to a query. The query can include one or more search terms, and the search engine can identify and rank the content items based on, for example, the search terms, e.g., keywords, in the query and one or more parameters associated with the content item. 15 [0004] In some online advertising systems, advertisers pay for their advertisements on a cost-per-click basis. Advertisers can select the maximum cost-per-click the advertisers are willing to pay for each click of an advertisement. The cost-per-click charged for an identified advertisement can be calculated based on the other advertisements rated or positioned below the current 20 advertisements and a click-through rate for the current advertisement in an auction process. [0005] Determining an actual cost-per-click for the advertisements that is not only based on the other advertisements (e.g., below the current advertisement) can result in an optimization of advertising revenue. Some 25 advertising systems charge the maximum cost-per-click. However, other optimization processes can also be used to select a subset of advertisements to be displayed and the actual cost-per-click to be charged. 1 WO 2009/061535 PCT/US2008/072665 SUMMARY [0006] Disclosed herein are systems, methods and computer program products for filtering and pricing content items. In one implementation, a threshold measure is determined for an advertisement based on one or more 5 parameters associated with the advertisement. A determination is made as to whether the threshold measure exceeds a filter threshold. The advertisement is promoted if the threshold measure exceeds the filter threshold, and an actual cost-per-click for the advertisement is based on the one or more parameters and the filter threshold if the advertisement is promoted. 10 [0007] In another implementation, one or more advertisements are received. The advertisements can each be associated with one or more parameters. Threshold measures are determined for each advertisement based on the one or more parameters, and the advertisements are filtered based on the threshold measures and a filter threshold. 15 [0008] In another implementation, a threshold measure is determined for an advertisement based on one or more parameters associated with the advertisement. A determination is made as to whether the threshold measure exceeds a filter threshold, and an actual cost-per-click is calculated for the advertisement if the threshold measure exceeds the filter threshold according to ?o the one or more parameters and the filter threshold. [0009] In another implementation, a first score associated with an advertisement is calculated. A determination is made as to whether an auction cost-per-click associated with the advertisement exceeds a reserve cost-per-click associated with the advertisement if the first score exceeds a threshold. An 25 advertiser associated with the advertisement is charged the auction cost-per-click if the first score exceeds the threshold and the auction cost-per-click exceeds the reserve cost-per-click. The advertiser is charged the reserve cost-per-click if the first score exceeds the threshold and the auction cost-per-click does not exceed the reserve cost-per-click. 30 [0010] In another implementation, a system includes a threshold engine that determines a threshold measure for an advertisement based on one or more 2 WO 2009/061535 PCT/US2008/072665 parameters associated with the advertisement, and determines if the threshold measure exceeds a filter threshold, and a pricing engine that calculates an actual cost-per-click for the advertisement if the threshold measure exceeds the filter threshold according to the one or more parameters. 5 BRIEF DESCRIPTION OF THE DRAWINGS [0011] Fig. 1 is a block diagram of an example implementation of an online advertising system. [0012] Fig. 2 is a block diagram of an example content threshold system. [0013] Fig. 3 is an example filtering and ranking table. 10 [0014] Fig. 4 is a flow diagram of an example process for filtering and pricing an advertisement. [0015] Fig. 5 is flow diagram of an example process for determining an actual cost-per-click for an advertisement. [0016] Fig. 6 is a flow diagram for another example process for filtering 15 and pricing an advertisement. [0017] Fig. 7 is a flow diagram of another example process for to filtering and pricing an advertisement. [0018] Fig. 8 is a schematic diagram of an example computer system that can be utilized to implement the systems and methods described herein. 20 DETAILED DESCRIPTION [0019] FIG. 1 is a block diagram of an example implementation of an online advertising system 100. In some implementations, one or more advertisers 102 can directly, or indirectly, enter, maintain, and track advertisement ("ad") information in an advertisement system 104. The 25 advertisements may be in the form of graphical advertisements, such as banner advertisements, text only advertisements, image advertisements, audio advertisements, video advertisements, advertisements combining one of more of any of such components, etc. The advertisements may also include embedded information, such as a links, meta-information, and/or machine executable 3 WO 2009/061535 PCT/US2008/072665 instructions. One or more publishers 106 may submit requests for advertisements to the system 104. The system 104 responds by sending advertisements (e.g., when an associated publication is rendered) to the requesting publisher 106 (or a browser associated with a requesting user) for 5 placement/co-location on one or more of the publisher's rendered web properties (e.g., websites and other network-distributed content). While reference is made to advertisements, other content items can be provided by the system 104. [0020] Other entities, such as users 108 and the advertisers 102, can provide usage information to the system 104, such as, for example, whether or 10 not a conversion or click-through related to an advertisement has occurred. [0021] A click-through can occur, for example, when a user of a user device, selects or "clicks" on an advertisement. The click-through rate can be a performance metric that is obtained by dividing the number of users that clicked on the advertisement or a link associated with the advertisement by the number 15 of times the advertisement was delivered. For example, if an advertisement is delivered 100 times, and three persons clicked on the advertisement, then the click-through rate for that advertisement is 3%. [0022] A "conversion" occurs when a user, for example, consummates a transaction related to a previously served advertisement. What constitutes a ?o conversion may vary from case to case and can be determined in a variety of ways. [0023] This usage information can include measured or observed user behavior related to advertisements that have been served. The system 104 performs financial transactions, such as crediting the publishers 106 and 25 charging the advertisers 102 based on the usage information. [0024] A computer network 110, such as a local area network (LAN), wide area network (WAN), the Internet, or a combination thereof, connects the advertisers 102, the system 104, the publishers 106, and the users 108. [0025] One example of a publisher 106 is a general content server that 30 receives requests for content (e.g., articles, discussion threads, music, video, graphics, search results, web page listings, information feeds, etc.), and retrieves 4 WO 2009/061535 PCT/US2008/072665 the requested content in response to the request. The content server may submit a request for advertisements to an advertisement server in the system 104. The advertisement request may include a number of advertisements desired. The advertisement request may also include content request 5 information. This information can include the content itself (e.g., page or other content document), a category corresponding to the content or the content request (e.g., arts, business, computers, arts-movies, arts-music, etc.), part or all of the content request, content age, content type (e.g., text, graphics, video, audio, mixed media, etc.), geo-location information, etc. 10 [0026] In some implementations, the content server can combine the requested content with one or more of the advertisements provided by the system 104. This combined content and advertisements can be sent to the user 108 that requested the content for presentation in a viewer (e.g., a browser or other content display system). The content server can transmit information about 15 the advertisements back to the advertisement server, including information describing how, when, and/or where the advertisements are to be rendered (e.g., in HTML or JavaScript T M ). [0027] Another example publisher 106 is a search service. A search service can receive queries for search results. In response, the search service ?o can retrieve relevant search results from an index of documents (e.g., from an index of web pages). An exemplary search service is described in the article S. Brin and L. Page, "The Anatomy of a Large-Scale Hypertextual Search Engine," Seventh International World Wide Web Conference, Brisbane, Australia and in U.S. Patent No. 6,285,999, both of which are incorporated herein by reference 25 each in their entirety. Search results can include, for example, lists of web page titles, snippets of text extracted from those web pages, and hypertext links to those web pages, and may be grouped into a predetermined number of (e.g., ten) search results. [0028] The search service can submit a request for advertisements to the 30 system 104. The request may include a number of advertisements desired. This number may depend on the search results, the amount of screen or page space 5 WO 2009/061535 PCT/US2008/072665 occupied by the search results, the size and shape of the advertisements, etc. In some implementations, the number of desired advertisements will be from one to ten, or from three to five. The request for advertisements may also include the query (as entered or parsed), information based on the query (such as geo 5 location information, whether the query came from an affiliate and an identifier of such an affiliate), and/or information associated with, or based on, the search results. Such information may include, for example, identifiers related to the search results (e.g., document identifiers or "dociDs"), scores related to the search results (e.g., information retrieval ("IR") scores), snippets of text extracted 10 from identified documents (e.g., web pages), full text of identified documents, feature vectors of identified documents, etc. In some implementations, IR scores can be computed from, for example, dot products of feature vectors corresponding to a query and a document, page rank scores, and/or combinations of IR scores and page rank scores, etc. 15 [0029] The search service can combine the search results with one or more of the advertisements provided by the system 104. This combined information can then be forwarded to the user 108 that requested the content. The search results can be maintained as distinct from the advertisements, so as not to confuse the user between paid advertisements and presumably neutral ?0 search results. [0030] Finally, the search service can transmit information about the advertisement and when, where, and/or how the advertisement was to be rendered back to the system 104. [0031] As can be appreciated from the foregoing, the advertising 25 management system 104 can serve publishers 106, such as content servers and search services. The system 104 permits serving of advertisements targeted to content (e.g., documents) served by content servers or in response to search queries provided by users. For example, a network or inter-network may include an advertisement server serving targeted advertisements in response to requests 30 from a search service with advertisement spots for sale. Suppose that the inter network is the World Wide Web. The search service crawls much or all of the 6 WO 2009/061535 PCT/US2008/072665 content. Some of this content will include advertisement spots (also referred to as "inventory") available. More specifically, one or more content servers may include one or more documents. Documents may include web pages, email, content, embedded information (e.g., embedded media), meta-information and 5 machine executable instructions, and advertisement spots available. The advertisements inserted into advertisement spots in a document can vary each time the document is served or, alternatively, can have a static association with a given document. [0032] In one implementation, the advertisement system 104 may include 10 an auction process to select advertisements. Advertisers may be permitted to select, or bid, an amount the advertisers are willing to pay for each click of an advertisement, e.g., a cost-per-click amount an advertiser pays when, for example, a user clicks on an advertisement. In one implementation, the cost per-click can include a maximum cost-per-click, e.g., the maximum amount the 15 advertiser is willing to pay for each click of an advertisement. For example, advertisers A, B, and C all select, or bid, a maximum cost-per-click of $1.00, $.60, and $.60, respectively. The maximum amount advertiser A will pay for a click is $1.00, the maximum amount advertiser B will pay is $.60, and the maximum amount advertiser C will pay is $.60. ?0 [0033] The position, or rank, of an advertisement, such as where the advertisement is displayed next to search results, can be a function of the cost per-click multiplied by a click-through rate associated with the advertisement. [0034] In one implementation, the rank of an advertisement can be determined by multiplying the maximum cost-per-click for the advertisement by 25 the click-through rate of the advertisement. The advertisement can then be placed among other advertisements in order of increasing or decreasing rank. For example, suppose the click-through rate of advertisers A, B, and C are "10%," "8%," and "3%," respectively. The rank of advertisements A, B, and C can be determined according to the max estimated cost-per-thousand impressions 30 (eCPM) of each advertisement. The max estimated cost-per-thousand impressions can be calculated as follows: 7 WO 2009/061535 PCT/US2008/072665 A: max eCPM = CTR x maximum cost-per-click = .10 x $1.00 .10 =.1 B: max eCPM = CTR x maximum cost-per-click = .08 x $.60 = .048 C: max eCPM = CTR x maximum cost-per-click = .03 x $.60 =.018 5 [0035] The advertisers can be ranked in decreasing order of max estimated cost-per-thousand impressions as follows: 1.A 2. B 10 3. C [0036] In some implementations, an advertisement may not be promoted unless the advertisement meets a threshold. Promoting an advertisement can, in some implementations, include locating the advertisement in a more visible 15 position. The advertisements can, for example, be placed above search results in response to a search query. In one implementation, promoting an advertisement includes presenting the advertisement to a user, where advertisements that are not promoted are not presented to the user. The advertisement can, for example, be promoted according to the following formula: 20 CTR * auction CPC > T [0037] In one implementation, the auction cost-per-click is the price that is necessary to keep the advertisement's position above the next advertisement. 25 To determine the auction cost-per-click, the system 104 can determine how much the advertiser in position 1 would have to pay to give them a rank equal to the advertiser in position 2. Then the system 104 adds a predetermined amount, e.g., $.01, to this amount. An auction cost-per-click of an advertisement can be determined based the click-through rate of the advertisement and maximum 30 estimated cost-per-thousand impressions of the advertisement below. To determine the auction cost-per-click, the system 104 can divide the maximum 8 WO 2009/061535 PCT/US2008/072665 estimated cost-per-thousand impressions of the advertisement below the current advertisement by the click-through rate of the current advertisement. Then the system 104 adds a predetermined amount, e.g., $.01, to this amount. The final amount is the auction CPC. 5 [0038] The last advertiser in the ranked list can pay a minimum cost-per click to hold the position in the list. For example, suppose the minimum cost-per click is $.20. The auction cost-per-click of advertisers A, B, and C can be determined as follows: 10 [0039] A: (B's max eCPM /A's click-through rate)/1 000 = (.048/.1) = $.48 + $.01 = $.49 [0040] B: (C's max eCPM / B's click-through rate)/1000 = (.018/.08) = $.23 + $.01= $.24 [0041] C: minimum cost-per-click = $.20 15 [0042] Therefore, A's auction cost-per-click is $.49, B's auction cost-per click is $.24, and C's auction cost-per-click is $.20. [0043] If the threshold was .02, then whether the advertisements were promoted according to the above formula CTR * auction CPC > T can be ?o determined as follows: [0044] A: .1 * $.49 = .049 > .02 [0045] B: .08 * $.24 = .0192 not greater than .02 [0046] C: .03 * $.20 = .006 not greater than .02 [0047] Therefore, in this example, advertisement A would be the only 25 advertisement promoted since it is the only advertisement that exceeds the threshold of .02. Thereafter, advertiser A is charged the auction cost-per-click amount of $.49. [0048] In some implementations, to maximize revenue, the advertisement system 104 can use a different formula to determine whether the threshold has 30 been met as well to determine an actual cost-per-click charged for each advertisement. 9 WO 2009/061535 PCT/US2008/072665 [0049] Fig. 2 is a block diagram of an example content filtering system 200. The content filtering system 200 can, for example, be implemented in a computer device or one or more computer devices connected through a network, e.g., a local area network (LAN) or a wide area network (WAN), such as the 5 Internet. The content filtering system 200 can, for example be implemented in the advertisement system 104, which can be implemented in a computing system. The one or more computing devices can, for example, include memory devices storing processing instructions and processing devices for executing the processing instructions. An example computing system is shown and described 10 with reference to Fig. 8. Other implementations, however, can also be used. [0050] In addition to filtering the content items to define a subset of content items eligible to be presented in a presentation environment, the content filtering system 200 can determine an actual cost-per-click to associate with each of the content items. 15 [0051] The content filtering system 200 can, for example, filter the content items to select a subset of the content items that have been ranked according to the auction process. The subset can be selected based on parameters associated with respective advertisements, e.g. cost-per-click and/or the click through rate of each advertisement. Only advertisements that are part of the ?0 selected subset can be displayed to a user in a presentation environment. [0052] The content filtering system 200 can, for example, include a filtering engine 202 and a content item data store 204. In one implementation, the content item data store 204 can comprise a unitary data store, such as a hard drive. In another implementation, the content data store 204 can comprise a 25 distributed data store, such as a storage system that is distributed over a network. Other implementations, however, can also be used. [0053] In one implementation, the content data store 204 can store one or more advertisements. Each advertisement in the content data store 204 can be associated with one or more parameters 206, 208, and 210. Each of the 30 parameters 206, 208, and 210 can be associated with performance metrics, e.g., click-through rates, conversions, auction metrics, cost-per-clicks, etc. In one 10 WO 2009/061535 PCT/US2008/072665 implementation, the content filtering system 200 can monitor and/or evaluate performance data related to the content items in the content data store 204. For example, the performance of each advertisement in the content data store 204 can be evaluated based on a performance metric associated with the 5 advertisement, such as a click-through rate, a conversion rate, or some other metric. In one implementation, the content filtering system 200 can also monitor and/or evaluate auction data related to the content items in the content data store 204. Each advertisement in the content data store 204 can be evaluated based on an auction metric associated with the advertisement, such as a cost-per-click. 10 [0054] In one implementation, the cost-per-click associated with an advertisement can be a maximum cost-per-click, an auction cost-per-click, or a reserve cost-per-click. The maximum cost-per-click, as described above, is the maximum amount an advertiser will pay for a click on their advertisement. The auction cost-per-click, as described above, is the price that is necessary to keep 15 the advertisement's position above the next advertisement. The reserve cost per-click is the minimum cost-per-click required for the advertisement to be promoted and displayed for a user. [0055] Other performance metrics can also be used, such as dwell time at a landing page, etc. The performance metrics can, for example, be revenue ?o related or non-revenue related. In another implementation, the performance metrics can be parsed according to time, e.g., the performance of a particular content item may be determined to be very high on weekends, moderate on weekday evenings, but very low on weekday mornings and afternoons, for example. 25 [0056] In one implementation, the filtering engine 202 can determine a threshold measure 212 for an advertisement. The threshold measure 212 can be used to determine whether the advertisement can be displayed (or promoted) to a user. The threshold measure 212 can be compared to a selected filter threshold 214 to determine whether the advertisement is selected as one of a 30 subset of advertisements. 11 WO 2009/061535 PCT/US2008/072665 [0057] In one implementation, the threshold measure 212 can be determined based on the one or more parameters 206, 208, and 210 associated with the advertisement. A threshold measure 212 can, for example, be determined using any one of the parameters 206, 208, 210 alone, or in 5 combination with other functions, e.g., an exponential function, and variables. The threshold measure 212 can, for example, be determined using the parameters click-through rate and cost-per-click. [0058] In another implementation, the threshold measure 212 can be determined by raising a quality score to an exponential power and multiplying the 10 quality score raised to an exponential power by a maximum cost-per click raised to an exponential power. For example, if the exponential values are "x" and "y," the threshold measure 212 can be determined according to the formula: [0059] Tadvertisement = QS_advertisementAx * 15 maxCPCadvertisementAy [0060] Where Tadvertisement is the threshold measure, QS_advertisement is the quality score of the advertisement, and maxCPCadvertisement is the maximum cost-per-click of the advertisement. [0061] A quality score can be the basis for measuring the quality and ?0 relevance of an advertisement and determining a minimum cost-per-click. The quality score can, for example, be determined by the advertisement's click through rate, the relevance of the advertisement text, overall historical keyword performance, and the user experience on a landing page associated with the advertisement. 25 [0062] In one implementation, the quality score can be calculated according to the formula: QS_advertisement = CTRadvertisementAa * GoodClickadvertisementAb 30 [0063] Where QS_advertisement is the quality score of the advertisement, CTRadvertisement is the click-through rate of the advertisement, and 12 WO 2009/061535 PCT/US2008/072665 GoodClickadvertisement is a prediction of whether a user will like the site associated with the advertisement if the user clicks on the advertisement. [0064] GoodClickadvertisement may be calculated, as described in co pending U.S. application Ser. No 10/321,064, entitled "Using estimated ad 5 qualities for ad filtering, ranking and promotion" and incorporated by reference herein. [0065] In another implementation, the threshold measure 212 can be determined according to the formula: 10 Tadvertisement = CTRadvertisementAx * maxCPCadvertisementAy [0066] Where Tadvertisement is the threshold measure, CTRadvertisement is the click-through rate of the advertisement, and maxCPCadvertisement is the maximum cost-per-click of the advertisement. 15 [0067] In one implementation, the filtering engine 202 can determine if the threshold measure 212 exceeds the filter threshold 214 and filter the advertisements based on the determination. [0068] In one implementation, the filtering engine 202 can, for example, determine the filter threshold 214 and the exponential value x and y based on ?o historical data associated with the advertisements, and by selecting the filter threshold 214 and the exponential values yielding, for example a desired or most promising simulation data. Historical data can, for example, include previous values of the parameters associated with the advertisements. For example, historical data can include previous click-through rates associated with the 25 advertisements. An advertisement can be associated with one or more click through rates based on previous performance. For example, the click-through rate of an advertisement can change and increase or decrease over time. Historical data can also include previous quality scores associated with the advertisements. An advertisement can be associated with one or more quality 30 scores based on previous performance. 13 WO 2009/061535 PCT/US2008/072665 [0069] Historical data can also include, for example, previous cost-per clicks associated with the advertisement. The cost-per-click of an advertisement can, for example, increase or decrease over time. An advertiser can, for example, decide whether to lower or increase the amount the advertiser would 5 pay for a click on the advertisement. [0070] Determining the filter threshold 214 and the exponential values based on historical data related to the advertisement can, for example, include generating simulation filter threshold values and simulation functional (e.g., exponential values for x and y). The filtering engine 202 can determine the filter 10 threshold 214 and the exponential values based on historical data related to the advertisement by generating simulation data based on simulation filter threshold values and the simulation exponential values, and selecting the filter threshold 214 and the exponential values yielding the most promising/desirable simulation data. The filtering engine 202 can, for example, use historical data associated 15 with an advertisement, such as previous click-through rates and cost-per-clicks, as well as simulation filter threshold values and simulation exponential values, to generate simulation data 216. The simulation filter threshold values can be used to simulate performance scenarios in which the threshold measure 212 is compared to the simulation filter threshold to determine if the simulated threshold ?0 measure 212 exceeds the simulation filter threshold, and generate expected revenue and quality estimates. An actual threshold value and exponential values x and y can be selected based on a maximized revenue value and/or maximized quality target. [0071] For example, a training set of historical data related to the 25 performance of advertisements that were not filtered can be utilized to generate simulation scenarios based on simulated filter thresholds and exponential values. The performance of the advertisements after filtering can be modeled to estimate a revenue or quality gain. For example, the changes in the probabilities of advertisements being selected in the absence of advertisements that were 30 actually selected can be modeled; the changes in the probabilities of 14 WO 2009/061535 PCT/US2008/072665 advertisements being selected in the presence of other advertisements that were not actually selected can be modeled; etc. [0072] In one implementation, the filtering engine 202 can compare the simulation data to the historical data, and optimize the filter threshold 214 and the 5 exponential values x and y based on the comparison. For example, the filtering engine 202 can perform iterative simulations to optimize the filter threshold based on the comparison. Optimizing the filter threshold can include, for example, adjusting the filter threshold 214 so that fewer or more advertisements are filtered. The iterative simulations can be utilized to estimate one or more of a 10 revenue gain or a quality gain. [0073] In one implementation, the value of the filter threshold 214 can be changed at any time. For example, the filter threshold can be changed weekly, monthly, bi-monthly, etc. In one implementation, the value of the filter threshold 214 can change according to budget usage of advertisers. For example, 15 advertisers can specify a budget to indicate the maximum amount of money to be spent on advertisements or campaigns for a certain amount of time. For example, a first advertiser may choose to pay a first amount per month for a campaign; a second advertiser may choose to pay a second amount per month for a campaign, etc. ?0 [0074] If advertiser budgets related to advertisements that are not filtered according to the filter threshold 214 are depleted, the amount of eligible advertisements for display may likewise be depleted. Accordingly, the filtering engine 202 can, for example, adjust the filter threshold 214 in this to allow a larger number of advertisements to become eligible for display. 25 [0075] In one implementation, the filtering engine 202 can promote the advertisement if the threshold measure 212 of the advertisement exceeds the filter threshold 214. Promoting an advertisement allows the filtering engine 202 to display the advertisement in a content item presentation environment 222. An advertisement that does not have a threshold measure 212 that exceeds the filter 30 threshold 214 may not be promoted, and will not be displayed in the environment 222. 15 WO 2009/061535 PCT/US2008/072665 [0076] In one implementation, the advertisement can be ranked according to the maximum estimated cost-per-thousand impressions of the advertisement after the advertisement is promoted. The maximum estimated cost-per-thousand impressions can be determined using the one or more parameters 206, 208, and 5 210. The maximum estimated cost-per-thousand impressions can, for example, be determined by multiplying a click-through rate associated with the advertisement by a maximum cost-per-click associated with the advertisement, as described above. In one implementation, this result can then be multiplied by 1000. The maximum estimated cost-per-thousand impressions can, for example, 10 be compared to maximum estimated cost-per-thousand impressions of other advertisements that also have a threshold measure 212 that exceeded the filter threshold 214, and the advertisements can be ranked by the ranking engine 220 according to the maximum estimated cost-per-thousand impressions. The ranked advertisements can be shown in a content item presentation environment 15 222, e.g., a web browser page. The advertisements can be ranked by each advertisement's maximum estimated cost-per-thousand impressions in decreasing order. Actions taken by a user in the presentation environment 222 can affect one or more of the parameters 206, 208 and 210, and accordingly affect subsequent rankings or presentations. ?0 [0077] In one implementation, the pricing engine 216 can determine an actual cost-per-click for the advertisement based on the one or more parameters 206, 208, and 210 and the filter threshold. The actual cost-per-click can be determined if the advertisement is promoted. The actual cost-per-click can, for example, be the amount the advertiser associated with the advertisement is 25 charged once the advertisement is promoted. [0078] In one implementation, determining the actual cost-per-click includes identifying the auction cost-per-click associated with the advertisement. The auction cost-per-click can be the price that is necessary to keep the advertisement's position above a next lower rated advertisement. To determine 30 the auction cost-per-click, the pricing engine 216 can determine how much the advertiser in position 1 would have to pay to give them a rank equal to the 16 WO 2009/061535 PCT/US2008/072665 advertiser in position 2. Then the pricing engine 216 adds a predetermined amount, e.g., $.01 to this amount. The auction cost-per-click of an advertisement can, for example, be determined based the click-through rate of the advertisement and maximum estimated cost-per-thousand impressions of the 5 advertisement below. To determine the auction cost-per-click, the pricing engine 216 can divide the maximum estimated cost-per-thousand impressions of the advertisement below the current advertisement by the click-through rate of the current advertisement and add a predetermined amount (e.g., $.01 is added) to this number. The final amount is the auction cost-per-click. 10 [0079] In one implementation, after the maximum estimated cost-per thousand impressions of the advertisement below the current advertisement is divided by the click-through rate of the current advertisement, this number is then divided by 1000. The pricing engine 216 can then add $.01 to this amount. In one implementation, if no advertiser exists below the current advertiser, then the 15 auction cost-per-click of the current advertiser is the minimum cost-per-click that is set by the pricing engine 216. In another implementation, the auction cost-per click can be calculated for every advertisement, whether or not the advertisement is promoted. In one implementation, if the auction cost-per-click is lower than the minimum cost-per-click set by the pricing engine 216, the advertiser is charged ?0 the minimum cost-per-click. [0080] In one implementation, determining the actual cost-per-click includes identifying the reserve cost-per-click associated with the advertisement. The reserve cost-per-click is a minimum amount required to promote an advertisement. The pricing engine 216 can calculate the reserve cost-per-click 25 by dividing the filter threshold 214 by the click-through rate of the advertisement. Then the pricing engine 216 adds a second predetermined amount (e.g., $.01) to this amount. This final amount is the reserve cost-per-click. [0081] In one implementation, the pricing engine 216 can select the greater of the auction cost-per-click and the reserve cost-per-click as the actual 30 cost-per-click for the advertisement. The advertiser 106 associated with the advertisement can be charged the greater of the auction cost-per-click and the 17 WO 2009/061535 PCT/US2008/072665 reserve cost-per-click, e.g., the actual cost-per-click, every time the advertisement of the advertiser 106 is clicked on by a user, as will be described below. [0082] Fig. 3 is an example filtering and ranking table 300. The table 300 5 describes the filtering and pricing of advertisements AD1, AD2 and AD3, and is based one or more parameters, e.g., a click-through rate, a maximum cost-per click, quality score, maximum estimated cost-per-thousand impressions, auction cost-per-click, a reserve cost-per-click, and an actual cost-per-click as indicated in the click-through rate column 302, the max CPC column 304, the QS column 10 306, the max eCPM column 308, the auction CPC column 310, the reserve CPC column 312, and the actual CPC column 314. [0083] As shown in Fig. 3, the advertisements AD1, AD2, and AD3 have corresponding click-through rates of 10%, 8%, and 3% respectively, corresponding maximum cost-per-click rates of $1.00, $.60, and $0.60, 15 respectively, and corresponding quality scores of .1, .08, and .03. A threshold measure, as listed in the threshold measure column 316, can be determined for each of the advertisements AD1 -AD3. The threshold measure can, for example, be determined based on the parameters quality score and maximum cost-per click parameters, e.g., QS Ax * maxCPCAy. In this example, x and y are "1." ?0 [0084] The filtering engine 202 can, for example, determine whether the threshold measure for each advertisement exceeds a filter threshold that is listed in a filter threshold column 318. Each of the threshold measures for AD1-AD3 can be compared against the filter threshold to determine which advertisements have threshold measures that exceed the filter threshold. In this example, the 25 advertisements AD1 and AD2 have threshold measures, e.g., 0.1 (.1A * 1.00A1) and .048 (.08^1 * .60A1), that exceed the filter threshold of .02, and the advertisement AD3 has a threshold measure, e.g., .018 (.03^1 * .60A1), that does not exceed the filter threshold of .02. Accordingly, the advertisements AD1 and AD2 are promoted, as indicated by the promote column 320. 30 [0085] In some implementations, the filtering engine 202 can rank the advertisements based on the one or more parameters. For example, the ranking 18 WO 2009/061535 PCT/US2008/072665 engine 220 can rank all of the advertisements based on the maximum eCPM (CTR * max CPC), the values for which are shown in the max eCPM column 308. In this example, the quality score is the same as the click-through rate of each advertisement. The advertisements AD1 - AD2 can be ranked accordingly. The 5 eCPM of AD1 is .1 (.10 * 1.00). The eCPM of AD2 is .048 (.08 * .60). Therefore AD1 is ranked first since its eCPM, e.g., .1, is higher than the eCPM of AD2, e.g., .048. However, as the advertisement AD3 is filtered out of the final presentation process because it did not have a threshold measure that exceeded the filter threshold, only the advertisements AD1 and AD2 are ranked and presented to a 10 user. The eCPM of AD3 can be calculated, however, because the eCPM is used to determine the auction cost-per-click for AD2, as described below. The eCPM of AD3 is .018 (.03 * .60). [0086] In some implementations, after the filtering engine 202 determines which advertisements have a threshold measure that exceeds the filter threshold 15 and therefore get promoted, the pricing engine 216 can determine an actual cost per-click of the advertisement based on the one or more parameters and the filter threshold. For example, the pricing engine 216 can calculate the auction cost per-click, as listed in column 310, and the reserve cost-per-click, as listed in column 312, and select the higher of the two as the actual cost-per-click, as listed ?0 in column 314. [0087] The pricing engine 216 can calculate the auction cost-per-click 310 of AD1 by, for example, dividing the eCPM of AD2 by the click-through rate of AD1. The pricing engine can then add a predetermined amount (e.g., $.01) to calculate the auction CPC. The auction cost-per-click of AD1 is $.49 ((.048/.1) + 25 $.01). The auction cost-per-click of AD2 can be calculated in the same manner. The auction cost-per-click of AD2 is $.24 ((.018/.08) + $.01). [0088] The pricing engine 216 can determine the reserve cost-per-click by dividing the filter threshold by the click-through rate of the advertisement and adding a predetermined amount ($.01). The reserve cost-per-click of AD1 is 30 calculated as $.21 ((.02/.1) + $.01). The reserve cost-per-click of AD2 is calculated as $.26 ((.02/.08) + $.01). 19 WO 2009/061535 PCT/US2008/072665 [0089] In one implementation, the pricing engine 216 can select the greater of the auction cost-per-click and the reserve cost-per-click as the actual cost-per-click to charge the advertiser of each advertisement. In this example, for AD1, the auction cost-per-click, e.g., $.49 is higher than the reserve cost-per 5 click, e.g., $.21. Therefore, the actual cost-per-click for AD1 is $.49, the auction cost-per-click. For AD2, the auction cost-per-click, e.g., $.24 is not higher than the reserve cost-per-click, e.g., $.26. Therefore, the actual cost-per-click of AD2 is $.26, the reserve cost-per-click. [0090] The filter threshold in the filter threshold column 318 may be 10 adjusted in response to a trigger event. For example, a trigger event can be a time period, e.g., monthly; a trigger event can be falling below a revenue target, e.g., the advertising system does not realize a weekly revenue goal; a trigger event can be based on an availability of a minimum number of advertisements; or some other event. 15 [0091] Fig. 4 is a flow diagram of an example process 400 for filtering and pricing an advertisement. The process 400 can, for example, be implemented in a system such as the content filtering system 200 of Fig. 2. [0092] Stage 402 determines a threshold measure for an advertisement based on one or more parameters associated with the advertisement. For ?o example, the filtering engine 202 can calculate a threshold measure for an advertisement based on one or more parameters associated with the advertisement, e.g., based on the calculation of QS Ax * maxCPCAy. Stage 404 determines if the threshold measure exceeds a filter threshold. For example, the filtering engine 202 can determine if the threshold measure exceeds a filter 25 threshold. Stage 406 promotes the advertisement if the threshold measure exceeds the filter threshold. For example, the filtering engine 220 can promote the advertisement if the threshold measure exceeds the filter threshold. Stage 408 determines an actual cost-per-click for the advertisement based on the one or more parameters and the filter threshold if the advertisement is promoted. For 30 example, the pricing engine can determine an actual cost-per-click for the 20 WO 2009/061535 PCT/US2008/072665 advertisement based on the one or more parameters and the filter threshold if the advertisement is promoted [0093] Fig. 5 is an example process 500 for determining an actual cost per-click for an advertisement based on the one or more parameters if the 5 advertisement is promoted. The process 500 can, for example, be implemented in a system such as the content filtering system 200 of Fig. 2. [0094] Stage 502 calculates the auction cost-per-click. For example, the pricing engine 216 can calculate the auction cost-per-click. Stage 504 calculates the reserve cost-per-click. For example, the pricing engine 216 can calculate the 10 reserve cost-per-click. Stage 506 selects the greater of the auction cost-per-click and the reserve cost-per-click as the actual cost-per-click for the advertisement. For example, the pricing engine 216 can select the greater of the auction cost per-click and the reserve cost-per-click as the actual cost-per-click for the advertisement. 15 [0095] Fig. 6 is another example process 600 for filtering and pricing an advertisement. The process 600 can, for example, be implemented in a system such as the content filtering system 200 of Fig. 2 [0096] Stage 602 determines a threshold measure for an advertisement based on one or more parameters associated with the advertisement. For ?o example, the filtering engine 202 can determine a threshold measure for an advertisement based on one or more parameters associated with the advertisement. Stage 604 determines if the threshold measure exceeds a filter threshold. For example, the filtering engine 202 can determine if the threshold measure exceeds a filter threshold. Stage 606 calculates an actual cost-per-click 25 for the advertisement if the threshold measure exceeds the filter threshold according to the one or more parameters and the filter threshold. For example, the pricing engine 216 can calculate an actual cost-per-click for the advertisement if the threshold measure exceeds the filter threshold according to the one or more parameters and the filter threshold. 21 WO 2009/061535 PCT/US2008/072665 [0097] Fig. 7 is another example process 700 for filtering and pricing an advertisement. The process 700 can, for example, be implemented in a system such as the content filtering system 200 of Fig. 2. [0098] Stage 702 calculates a first score associated with an 5 advertisement. For example, the pricing engine 216 can calculate a first score associated with an advertisement. Stage 704 determines whether an auction cost-per-click associated with the advertisement exceeds a reserve cost-per-click associated with the advertisement (e.g., if the first score exceeds a threshold). For example, the pricing engine 216 can determine whether an auction cost-per 10 click associated with the advertisement exceeds a reserve cost-per-click associated with the advertisement if the first score exceeds a threshold. Stage 706 charges an advertiser associated with the advertisement the auction cost per-click if the first score exceeds the threshold and the auction cost-per-click exceeds the reserve cost-per-click. For example, the pricing engine 216 can 15 charge an advertiser associated with the advertisement the auction cost-per-click if the first score exceeds the threshold and the auction cost-per-click exceeds the reserve cost-per-click. Stage 708 charges the advertiser the reserve cost-per click if the first score exceeds the threshold and the auction cost-per-click does not exceed the reserve cost-per-click. For example, the pricing engine 216 can ?0 charge the advertiser the reserve cost-per-click if the first score exceeds the threshold and the auction cost-per-click does not exceed the reserve cost-per click. [0099] Fig. 8 is block diagram of an example computer system 800. The system 800 includes a processor 810, a memory 820, a storage device 830, and 25 an input/output device 840. Each of the components 810, 820, 830, and 840 can, for example, be interconnected using a system bus 850. The processor 810 is capable of processing instructions for execution within the system 800. In one implementation, the processor 810 is a single-threaded processor. In another implementation, the processor 810 is a multi-threaded processor. The processor 30 810 is capable of processing instructions stored in the memory 820 or on the storage device 830. 22 WO 2009/061535 PCT/US2008/072665 [0100] The memory 820 stores information within the system 800. In one implementation, the memory 820 is a computer-readable medium. In one implementation, the memory 820 is a volatile memory unit. In another implementation, the memory 820 is a non-volatile memory unit. 5 [0101] The storage device 830 is capable of providing mass storage for the system 800. In one implementation, the storage device 830 is a computer readable medium. In various different implementations, the storage device 830 can, for example, include a hard disk device, an optical disk device, or some other large capacity storage device. 10 [0102] The input/output device 840 provides input/output operations for the system 800. In one implementation, the input/output device 840 can include one or more of a network interface devices, e.g., an Ethernet card, a serial communication device, e.g., and RS-232 port, and/or a wireless interface device, e.g., and 802.11 card. In another implementation, the input/output device can 15 include driver devices configured to receive input data and send output data to other input/output devices, e.g., keyboard, printer and display devices 860. Other implementations, however, can also be used, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc. [0103] Although the above description refers to a content item such as an ?o advertisement, content items such as video and/or audio files, web pages for particular subjects, news articles, etc. can also be used. For example, if a user clicks on a video file, then the owner or publisher of the video file can also generate revenue every time a user clicks on the video file. A threshold measure can also be determined for the video file according to one or more parameters 25 associated with the video file, e.g., a click-through rate and/or a cost-per-click of the video file. [0104] The apparatus, methods, flow diagrams, and structure block diagrams described in this patent document may be implemented in computer processing systems including program code comprising program instructions that 30 are executable by the computer processing system. Other implementations may also be used. Additionally, the flow diagrams and structure block diagrams 23 described in this patent document, which describe particular methods and/or corresponding acts in support of steps and corresponding functions in support of disclosed structural means, may also be utilized to implement corresponding software structures and algorithms, and equivalents thereof. [0105] This written description sets forth the best mode of the invention and provides examples to describe the invention and to enable a person of ordinary skill in the art to make and use the invention. This written description does not limit the invention to the precise terms set forth. Thus, while the invention has been described in detail with reference to the examples set forth above, those of ordinary skill in the art may affect alterations, modifications and variations to the examples without departing from the scope of the invention. [0106] Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. [0107] The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge in Australia. 24

Claims (7)

1. A method, comprising: calculating a first score associated with an advertisement; determining whether an auction cost-per-click associated with the advertisement exceeds a reserve cost-per-click associated with the advertisement if the first score exceeds a threshold; charging an advertiser associated with the advertisement the auction cost-per click if the first score exceeds the threshold and the auction cost-per-click exceeds the reserve cost-per-click; and charging the advertiser the reserve cost-per-click if the first score exceeds the threshold and the auction cost-per-click does not exceed the reserve cost-per-click.

2. The method of claim 1, wherein the first score is based one or more parameters associated with the advertisement.

3. The method of claim 2, wherein at least one parameter is a quality score.

4. The method of claim 3, wherein at least one parameter is a cost-per click.

5. The method of claim 4, wherein the cost-per-click is one of a maximum cost-per-click, an auction cost-per-click, and a reserve cost-per-click.

6. The method of claim 4, wherein calculating a first score associated with an advertisement comprises: multiplying the maximum cost-per-click raised to a first exponential power and the quality score raised to a second exponential power.

7. The method of claim 5 , wherein determining whether an auction cost per-click associated with the advertisement exceeds a reserve cost-per-click associated with the advertisement if the first score exceeds a threshold comprises: 25 identifying the auction cost-per click based on the one or more parameters; and identifying the reserve cost-per click based on the one or more parameters. 26