Today I challenge our CEP/ESP/EP vendors (or SIs) to create the following solution to detect and block rogue bots on Apache web sites.   I will install and test each submitted solution on The UNIX Forums and post the results here.

Here are some basic requirements:

1. Your solution must run on Linux and be installable and configurable remotely with SSH or HTTP.  There will be no physical access to the server. No exceptions.
2. Preferrably, the configuration can be done with a Web-Based Interface (WBI) - a browser.
3. Your solution will listen to continuous updates to the Apache2 access log, exact location configurable in your solution, and identify robots ( bots), also known as spiders, from the log.
4. Your solution will provide a confidence metric, key indicator (KI), for each bot detected, from 0 to 10, where 10 indicates “absolutely a bot,” 0 is “absolutely not a bot.”
5. Your solution will update the IP address of each bot and KI you identify in a file/table called, for example, ./bot_scorecard.txt where each line is an IP address of a bot, followed by a semicolon (or other delimiter of your choice) and the confidence factor, for example,  10.0.0.1;10 means that 10.0.0.1 is a bot, 100% sure.
6. Your solution must compare bots detected to a file/table called, for example, ./bots_allowed.txt and ./bots_denied.txt that are in the format IP address/mask, for example 10.0.0.1/24, or 10.0.0.1/32.
7. If the KI “confidence factor” of the IP address of your detected bot is higher than the tunable “is a bot” KI, then your solution should update the tables/files and then call iptables and block the bot.
8. It should send an email to one or more email addresses with a message, for example:  “New Bot Detected - Confidence 8″ with IP address, etc. in the message.  Another example would be an email, “Bot Blocked” - with details, etc.
9. You cannot automatically block any traffic that is not a bot.  Blocking one “non-bot” results in failure, no exceptions.
10. The Prize:  The winner will get their logo (w/link) on this site in a block called “Bot Hunter Winner” (or something like that.)

These are some basic requirements; I don’t want to restrict your thinking or solution, so be creative!  Feel free to ask any questions in the comment section of this thread.

Remember, sometimes you may have to manage the state of IP addresses for days, or hours, before you can accurately deterimine if it is a bot based on behavior alone.   So, you will need to work with both long and short time windows.  Latency is not important. Detection accurate is importance.

Anyone care to submit a solution for testing?

Blogger: Ramon Krikken

The web browser is one of those peculiar pieces of software, having to accept input from arbitrary sources and then parse and render the data that is sent to it. Part of this it does by itself, and other parts are taken care of by handlers and plug-ins. In doing so, it displays hypertext, images, videos, and even runs active content like Flash, JavaScript, and ActiveX.

But however much we love the browser, we’ve also come to hate the myriad of vulnerabilities that affect it. Everything from cross-site scripting to remote code execution via maliciously formed animated cursor files and Flash content can make browsing a hazardous activity. The browser is sick, and that’s not desirable for a platform we use for important business and personal transactions.

Worsening the browser’s diagnosis is the recent paper from Mark Dowd and Alexander Sotirov, sub-titled “Setting back browser security by 10 years,” which discusses how to bypass Microsoft Vista’s memory protection capabilities with some added effort for the exploit designers. It’s not that all of the techniques are necessarily new, but the browser appears to be particularly vulnerable to easy exploitation.

Surprising? Not exactly, when we take into account that the browser is suffering from the same disease as the general purpose operating system: bloat and compatibility. We expect the browser to do ever more, but everything we used it for before still needs to work as if it were yesterday. It feels a bit like people insisting on using a cardboard box as a safe, and wondering why their money keeps getting stolen.

It’s not like we haven’t been working on the browser’s cure, though. There have been some improvements in the browsers themselves, the operating systems have also implemented compensating controls, but most of all, there has been an enormous push for securing the web applications that deliver the data in the first place. Unfortunately, the latter two won’t help secure the browser in the long run.

The first issue is that not all content will come from ‘nice’ servers, the second that the server can only make an educated guess on how a browser will parse and render a given set of data, and the third that operating system controls have their own limitations, whether by design or implementation (for example needing to re-compile existing code to enable certain protections.) The browser, in the end, has to be mostly responsible for keeping itself safe; the operating system must assist it in doing so.

So we’re in a pickle. The browser is sick (and the operating system is too), but it’s hard to cure it without a redesign that will undoubtedly impact compatibility, the ever-so-desired multi-functionality, or its ease of use. We can layer defenses by using web filtering in the enterprise environment, but in the end – for the consumer market in particular – we need to fix the browser itself. I can think of a few things I think might help:

• Some kind of site security policy  to restrict where the browser loads auxiliary content from, and which data it can ‘trust’, when loading a web page (I’d prefer mandatory enforcement, and adding an HTML tag to be able to indicate blocks of untrustworthy data.)
• Restricted compartments for plug-ins to run in, ensuring that their bugs cannot easily affect the whole browser.
• Better software development practices for the plug-ins and content parsers themselves, so that they’re less vulnerable, and compiled with the latest protection measures to begin with.

All of this means more work, and some of it means a lot of unhappy reactions when things stop working. Even then we will of course still have to deal with additional vulnerabilities, such as those that may be present in hardware, but we will at least have taken prudent steps to ‘find a cure.’

Blogger: Ramon Krikken

The web browser is one of those peculiar pieces of software, having to accept input from arbitrary sources and then parse and render the data that is sent to it. Part of this it does by itself, and other parts are taken care of by handlers and plug-ins. In doing so, it displays hypertext, images, videos, and even runs active content like Flash, JavaScript, and ActiveX.

But however much we love the browser, we???ve also come to hate the myriad of vulnerabilities that affect it. Everything from cross-site scripting to remote code execution via maliciously formed animated cursor files and Flash content can make browsing a hazardous activity. The browser is sick, and that???s not desirable for a platform we use for important business and personal transactions.

Worsening the browser???s diagnosis is the recent paper from Mark Dowd and Alexander Sotirov, sub-titled ???Setting back browser security by 10 years,??? which discusses how to bypass Microsoft Vista???s memory protection capabilities with some added effort for the exploit designers. It???s not that all of the techniques are necessarily new, but the browser appears to be particularly vulnerable to easy exploitation.

Surprising? Not exactly, when we take into account that the browser is suffering from the same disease as the general purpose operating system: bloat and compatibility. We expect the browser to do ever more, but everything we used it for before still needs to work as if it were yesterday. It feels a bit like people insisting on using a cardboard box as a safe, and wondering why their money keeps getting stolen.

It???s not like we haven???t been working on the browser???s cure, though. There have been some improvements in the browsers themselves, the operating systems have also implemented compensating controls, but most of all, there has been an enormous push for securing the web applications that deliver the data in the first place. Unfortunately, the latter two won???t help secure the browser in the long run.

The first issue is that not all content will come from ???nice??? servers, the second that the server can only make an educated guess on how a browser will parse and render a given set of data, and the third that operating system controls have their own limitations, whether by design or implementation (for example needing to re-compile existing code to enable certain protections.) The browser, in the end, has to be mostly responsible for keeping itself safe; the operating system must assist it in doing so.

So we???re in a pickle. The browser is sick (and the operating system is too), but it???s hard to cure it without a redesign that will undoubtedly impact compatibility, the ever-so-desired multi-functionality, or its ease of use. We can layer defenses by using web filtering in the enterprise environment, but in the end ??? for the consumer market in particular ??? we need to fix the browser itself. I can think of a few things I think might help:

• Some kind of site security policy  to restrict where the browser loads auxiliary content from, and which data it can ???trust???, when loading a web page (I???d prefer mandatory enforcement, and adding an HTML tag to be able to indicate blocks of untrustworthy data.)
• Restricted compartments for plug-ins to run in, ensuring that their bugs cannot easily affect the whole browser.
• Better software development practices for the plug-ins and content parsers themselves, so that they???re less vulnerable, and compiled with the latest protection measures to begin with.

All of this means more work, and some of it means a lot of unhappy reactions when things stop working. Even then we will of course still have to deal with additional vulnerabilities, such as those that may be present in hardware, but we will at least have taken prudent steps to ???find a cure.???

Dear Registrar, This message is intended to explain how certain decisions that were made by the ICANN Board of Directors at its meeting in Paris last week may affect your registrar. Specifically, the Board adopted GNSO recommendations on domain tasting that included both budget and non-budget provisions designed to restrict the applicability of the Add Grace Period (AGP). Please note that this message is a summary of changes that affect registrars. You should refer to the adopted budget document and adopted motions for further information. Summary of Important Timing Issues After several months of discussion and public comment on both the budget and the GNSO recommendations, the Board has approved the proposed budget containing a provision for collecting transaction fees above a threshold during the AGP. Effective 1 July 2008, the registrar-level transaction fee will be collected on transactions, including names added on or after 1 July 2008 and deleted during the Add Grace Period above a certain minimum threshold. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. The budget assumes the transaction fee rate will remain at US ./send.20. The second change prohibits registries from issuing refunds above a similar threshold for names registered and deleted during the AGP (although some registries have made plans to charge for such transactions independent of this motion). The implementation timing of this change has not been set, but should be expected to take place over a period of some months. ICANN staff will solicit public comments and post a registrar advisory prior to implementation of this aspect of the GNSO recommendation. Budget - Registrar Fees Effective 1 July 2008 The Operating Plan and Budget details for 2008-2009 fiscal year can be found at: http://www.icann.org/en/financials/proposed-opplan-budget-v3-fy09-25jun0 8-en.pdf Relevant section from the approved budget: * Registrar-Level Transaction Fees In FY08 the per transaction-year rate was ./send.20 (or a 5 cent discount from the established ./send.25 rate). The draft FY09 budget assumes that the ./send.20 rate will continue for registrar transaction fees. As in past years, each transaction will be defined as one-year domain registration increment caused by a successful add renewal or transfer command. FY09 revenue is estimated to be .4 million for registrar-level transaction fees. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. Therefore per-transaction fee will continue to be charged for each one-year increment of every transaction (e.g. at a ./send.20 fee level, the fee for a three-year renewal will be US ./send.60), and registrars will continue to have the option to "defer" payment of the fees for the years beyond one for each transaction. n Note, as in previous years, ICANN can collect such fees directly from the registrars only if they are "expressly approved by registrars who account, in the aggregate, for payment of two-thirds of all registrar-level fees collected by ICANN." ICANN will shortly undertake the process of requesting such approval for the 2008-09 fiscal year. While ICANN is grateful for consistent approval by registrars of fee levels in prior years, and is optimistic about such approval this year, if for some reason the necessary approval is not achieved, the fees will be collected by ICANN, as permitted under the registry agreements through the registries. (Note that the amount of such fees varies by registry, but in no case exceeds US ./send.25.) Registries will then be able to collect those payments from registrars to the extent permitted under the relevant contracts. It is expected that the same transaction increments (including AGP) will be covered, whether collected directly by ICANN or in! directly by the registries, so registrars should anticipate this liability under either scenario. ICANN Board Resolution Whereas, ICANN community stakeholders are increasingly concerned about domain tasting, which is the practice of using the add grace period (AGP) to register domain names in bulk in order to test their profitability. Whereas, on 17 April 2008, the GNSO Council approved, by a Supermajority vote, a motion to prohibit any gTLD operator that has implemented an AGP from offering a refund for any domain name deleted during the AGP that exceeds 10% of its net new registrations in that month, or fifty domain names, whichever is greater. Whereas, on 25 April 2008, the GNSO Council forwarded its formal "Report to the ICANN Board - Recommendation for Domain Tasting" , which outlines the full text of the motion and the full context and procedural history of this proceeding. Whereas, the Board is also considering the Proposed FY 09 Operating Plan and Budget , which includes (at the encouragement of the GNSO Council) a proposal similar to the GNSO policy recommendation to expand the applicability of the ICANN transaction fee in order to limit domain tasting. Resolved (2008.06.26.06), the Board adopts the GNSO policy recommendation on domain tasting, and directs staff to implement the policy following appropriate comment and notice periods on the implementation documents. Domain tasting motion approved by the GNSO Council 17 April 2008 Whereas, the GNSO Council has discussed the Issues Report on Domain Tasting and the Final Outcomes Report of the ad hoc group on Domain Tasting; Whereas, the GNSO Council resolved on 31 October 2007 to launch a PDP on Domain Tasting; Whereas, the GNSO Council authorized on 17 January 2008 the formation of a small design team to develop a plan for the deliberations on the Domain Tasting PDP (the "Design Team"), the principal volunteers to which had been members of the Ad Hoc Group on Domain Tasting and were well-informed of both the Final Outcomes Report of the Ad Hoc Group on Domain Tasting and the GNSO Initial Report on Domain Tasting (collectively with the Issues Report, the "Reports on Domain Tasting"); Whereas, the GNSO Council has received the Draft Final Report on Domain Tasting; Whereas, PIR, the .org registry operator, has amended its Registry Agreement to charge an Excess Deletion Fee; and both NeuStar, the .biz registry operator, and Afilias, the .info registry operator, are seeking amendments to their respective Registry Agreements to modify the existing AGP; The GNSO Council recommends to the ICANN Board of Directors that: 1. The applicability of the Add Grace Period shall be restricted for any gTLD which has implemented an AGP ("Applicable gTLD Operator"). Specifically, for each Applicable gTLD Operator: a. During any given month, an Applicable gTLD Operator may not offer any refund to a registrar for any domain names deleted during the AGP that exceed (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. b. A Registrar may seek an exemption from the application of such restriction in a specific month, upon the documented showing of extraordinary circumstances. For any Registrar requesting such an exemption, the Registrar must confirm in writing to the Registry Operator how, at the time the names were deleted, these extraordinary circumstances were not known, reasonably could not have been known, and were outside of the Registrar's control. Acceptance of any exemption will be at the sole reasonable discretion of the Registry Operator, however "extraordinary circumstances" which reoccur regularly will not be deemed extraordinary. c. In addition to all other reporting requirements to ICANN, each Applicable gTLD Operator shall identify each Registrar that has sought an exemption, along with a brief descriptive identification of the type of extraordinary circumstance and the action (if any) that was taken by the Applicable gTLD Operator. 2. Implementation and execution of these recommendations shall be monitored by the GNSO. Specifically; a. ICANN Staff shall analyze and report to the GNSO at six month intervals for two years after implementation, until such time as the GNSO resolves otherwise, with the goal of determining; i. How effectively and to what extent the policies have been implemented and followed by Registries and Registrars, and ii. Whether or not modifications to these policies should be considered by the GNSO as a result of the experiences gained during the implementation and monitoring stages, b. The purpose of these monitoring and reporting requirements are to allow the GNSO to determine when, if ever, these recommendations and any ensuing policy require additional clarification or attention based on the results of the reports prepared by ICANN Staff.

Dear Registrar, This message is intended to explain how certain decisions that were made by the ICANN Board of Directors at its meeting in Paris last week may affect your registrar. Specifically, the Board adopted GNSO recommendations on domain tasting that included both budget and non-budget provisions designed to restrict the applicability of the Add Grace Period (AGP). Please note that this message is a summary of changes that affect registrars. You should refer to the adopted budget document and adopted motions for further information. Summary of Important Timing Issues After several months of discussion and public comment on both the budget and the GNSO recommendations, the Board has approved the proposed budget containing a provision for collecting transaction fees above a threshold during the AGP. Effective 1 July 2008, the registrar-level transaction fee will be collected on transactions, including names added on or after 1 July 2008 and deleted during the Add Grace Period above a certain minimum threshold. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. The budget assumes the transaction fee rate will remain at US ./send.20. The second change prohibits registries from issuing refunds above a similar threshold for names registered and deleted during the AGP (although some registries have made plans to charge for such transactions independent of this motion). The implementation timing of this change has not been set, but should be expected to take place over a period of some months. ICANN staff will solicit public comments and post a registrar advisory prior to implementation of this aspect of the GNSO recommendation. Budget - Registrar Fees Effective 1 July 2008 The Operating Plan and Budget details for 2008-2009 fiscal year can be found at: http://www.icann.org/en/financials/proposed-opplan-budget-v3-fy09-25jun0 8-en.pdf Relevant section from the approved budget: * Registrar-Level Transaction Fees In FY08 the per transaction-year rate was ./send.20 (or a 5 cent discount from the established ./send.25 rate). The draft FY09 budget assumes that the ./send.20 rate will continue for registrar transaction fees. As in past years, each transaction will be defined as one-year domain registration increment caused by a successful add renewal or transfer command. FY09 revenue is estimated to be .4 million for registrar-level transaction fees. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. Therefore per-transaction fee will continue to be charged for each one-year increment of every transaction (e.g. at a ./send.20 fee level, the fee for a three-year renewal will be US ./send.60), and registrars will continue to have the option to "defer" payment of the fees for the years beyond one for each transaction. n Note, as in previous years, ICANN can collect such fees directly from the registrars only if they are "expressly approved by registrars who account, in the aggregate, for payment of two-thirds of all registrar-level fees collected by ICANN." ICANN will shortly undertake the process of requesting such approval for the 2008-09 fiscal year. While ICANN is grateful for consistent approval by registrars of fee levels in prior years, and is optimistic about such approval this year, if for some reason the necessary approval is not achieved, the fees will be collected by ICANN, as permitted under the registry agreements through the registries. (Note that the amount of such fees varies by registry, but in no case exceeds US ./send.25.) Registries will then be able to collect those payments from registrars to the extent permitted under the relevant contracts. It is expected that the same transaction increments (including AGP) will be covered, whether collected directly by ICANN or in! directly by the registries, so registrars should anticipate this liability under either scenario. ICANN Board Resolution Whereas, ICANN community stakeholders are increasingly concerned about domain tasting, which is the practice of using the add grace period (AGP) to register domain names in bulk in order to test their profitability. Whereas, on 17 April 2008, the GNSO Council approved, by a Supermajority vote, a motion to prohibit any gTLD operator that has implemented an AGP from offering a refund for any domain name deleted during the AGP that exceeds 10% of its net new registrations in that month, or fifty domain names, whichever is greater. Whereas, on 25 April 2008, the GNSO Council forwarded its formal "Report to the ICANN Board - Recommendation for Domain Tasting" , which outlines the full text of the motion and the full context and procedural history of this proceeding. Whereas, the Board is also considering the Proposed FY 09 Operating Plan and Budget , which includes (at the encouragement of the GNSO Council) a proposal similar to the GNSO policy recommendation to expand the applicability of the ICANN transaction fee in order to limit domain tasting. Resolved (2008.06.26.06), the Board adopts the GNSO policy recommendation on domain tasting, and directs staff to implement the policy following appropriate comment and notice periods on the implementation documents. Domain tasting motion approved by the GNSO Council 17 April 2008 Whereas, the GNSO Council has discussed the Issues Report on Domain Tasting and the Final Outcomes Report of the ad hoc group on Domain Tasting; Whereas, the GNSO Council resolved on 31 October 2007 to launch a PDP on Domain Tasting; Whereas, the GNSO Council authorized on 17 January 2008 the formation of a small design team to develop a plan for the deliberations on the Domain Tasting PDP (the "Design Team"), the principal volunteers to which had been members of the Ad Hoc Group on Domain Tasting and were well-informed of both the Final Outcomes Report of the Ad Hoc Group on Domain Tasting and the GNSO Initial Report on Domain Tasting (collectively with the Issues Report, the "Reports on Domain Tasting"); Whereas, the GNSO Council has received the Draft Final Report on Domain Tasting; Whereas, PIR, the .org registry operator, has amended its Registry Agreement to charge an Excess Deletion Fee; and both NeuStar, the .biz registry operator, and Afilias, the .info registry operator, are seeking amendments to their respective Registry Agreements to modify the existing AGP; The GNSO Council recommends to the ICANN Board of Directors that: 1. The applicability of the Add Grace Period shall be restricted for any gTLD which has implemented an AGP ("Applicable gTLD Operator"). Specifically, for each Applicable gTLD Operator: a. During any given month, an Applicable gTLD Operator may not offer any refund to a registrar for any domain names deleted during the AGP that exceed (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. b. A Registrar may seek an exemption from the application of such restriction in a specific month, upon the documented showing of extraordinary circumstances. For any Registrar requesting such an exemption, the Registrar must confirm in writing to the Registry Operator how, at the time the names were deleted, these extraordinary circumstances were not known, reasonably could not have been known, and were outside of the Registrar's control. Acceptance of any exemption will be at the sole reasonable discretion of the Registry Operator, however "extraordinary circumstances" which reoccur regularly will not be deemed extraordinary. c. In addition to all other reporting requirements to ICANN, each Applicable gTLD Operator shall identify each Registrar that has sought an exemption, along with a brief descriptive identification of the type of extraordinary circumstance and the action (if any) that was taken by the Applicable gTLD Operator. 2. Implementation and execution of these recommendations shall be monitored by the GNSO. Specifically; a. ICANN Staff shall analyze and report to the GNSO at six month intervals for two years after implementation, until such time as the GNSO resolves otherwise, with the goal of determining; i. How effectively and to what extent the policies have been implemented and followed by Registries and Registrars, and ii. Whether or not modifications to these policies should be considered by the GNSO as a result of the experiences gained during the implementation and monitoring stages, b. The purpose of these monitoring and reporting requirements are to allow the GNSO to determine when, if ever, these recommendations and any ensuing policy require additional clarification or attention based on the results of the reports prepared by ICANN Staff.

Dear Registrar, This message is intended to explain how certain decisions that were made by the ICANN Board of Directors at its meeting in Paris last week may affect your registrar. Specifically, the Board adopted GNSO recommendations on domain tasting that included both budget and non-budget provisions designed to restrict the applicability of the Add Grace Period (AGP). Please note that this message is a summary of changes that affect registrars. You should refer to the adopted budget document and adopted motions for further information. Summary of Important Timing Issues After several months of discussion and public comment on both the budget and the GNSO recommendations, the Board has approved the proposed budget containing a provision for collecting transaction fees above a threshold during the AGP. Effective 1 July 2008, the registrar-level transaction fee will be collected on transactions, including names added on or after 1 July 2008 and deleted during the Add Grace Period above a certain minimum threshold. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. The budget assumes the transaction fee rate will remain at US ./send.20. The second change prohibits registries from issuing refunds above a similar threshold for names registered and deleted during the AGP (although some registries have made plans to charge for such transactions independent of this motion). The implementation timing of this change has not been set, but should be expected to take place over a period of some months. ICANN staff will solicit public comments and post a registrar advisory prior to implementation of this aspect of the GNSO recommendation. Budget - Registrar Fees Effective 1 July 2008 The Operating Plan and Budget details for 2008-2009 fiscal year can be found at: http://www.icann.org/en/financials/proposed-opplan-budget-v3-fy09-25jun0 8-en.pdf Relevant section from the approved budget: * Registrar-Level Transaction Fees In FY08 the per transaction-year rate was ./send.20 (or a 5 cent discount from the established ./send.25 rate). The draft FY09 budget assumes that the ./send.20 rate will continue for registrar transaction fees. As in past years, each transaction will be defined as one-year domain registration increment caused by a successful add renewal or transfer command. FY09 revenue is estimated to be .4 million for registrar-level transaction fees. Each "transaction" will continue to be defined as a one-year domain registration increment caused by a successful add, renewal or transfer command, but this year any domain names deleted during the AGP (if offered) will be included as transactions if they exceed the maximum of (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. Therefore per-transaction fee will continue to be charged for each one-year increment of every transaction (e.g. at a ./send.20 fee level, the fee for a three-year renewal will be US ./send.60), and registrars will continue to have the option to "defer" payment of the fees for the years beyond one for each transaction. n Note, as in previous years, ICANN can collect such fees directly from the registrars only if they are "expressly approved by registrars who account, in the aggregate, for payment of two-thirds of all registrar-level fees collected by ICANN." ICANN will shortly undertake the process of requesting such approval for the 2008-09 fiscal year. While ICANN is grateful for consistent approval by registrars of fee levels in prior years, and is optimistic about such approval this year, if for some reason the necessary approval is not achieved, the fees will be collected by ICANN, as permitted under the registry agreements through the registries. (Note that the amount of such fees varies by registry, but in no case exceeds US ./send.25.) Registries will then be able to collect those payments from registrars to the extent permitted under the relevant contracts. It is expected that the same transaction increments (including AGP) will be covered, whether collected directly by ICANN or in! directly by the registries, so registrars should anticipate this liability under either scenario. ICANN Board Resolution Whereas, ICANN community stakeholders are increasingly concerned about domain tasting, which is the practice of using the add grace period (AGP) to register domain names in bulk in order to test their profitability. Whereas, on 17 April 2008, the GNSO Council approved, by a Supermajority vote, a motion to prohibit any gTLD operator that has implemented an AGP from offering a refund for any domain name deleted during the AGP that exceeds 10% of its net new registrations in that month, or fifty domain names, whichever is greater. Whereas, on 25 April 2008, the GNSO Council forwarded its formal "Report to the ICANN Board - Recommendation for Domain Tasting" , which outlines the full text of the motion and the full context and procedural history of this proceeding. Whereas, the Board is also considering the Proposed FY 09 Operating Plan and Budget , which includes (at the encouragement of the GNSO Council) a proposal similar to the GNSO policy recommendation to expand the applicability of the ICANN transaction fee in order to limit domain tasting. Resolved (2008.06.26.06), the Board adopts the GNSO policy recommendation on domain tasting, and directs staff to implement the policy following appropriate comment and notice periods on the implementation documents. Domain tasting motion approved by the GNSO Council 17 April 2008 Whereas, the GNSO Council has discussed the Issues Report on Domain Tasting and the Final Outcomes Report of the ad hoc group on Domain Tasting; Whereas, the GNSO Council resolved on 31 October 2007 to launch a PDP on Domain Tasting; Whereas, the GNSO Council authorized on 17 January 2008 the formation of a small design team to develop a plan for the deliberations on the Domain Tasting PDP (the "Design Team"), the principal volunteers to which had been members of the Ad Hoc Group on Domain Tasting and were well-informed of both the Final Outcomes Report of the Ad Hoc Group on Domain Tasting and the GNSO Initial Report on Domain Tasting (collectively with the Issues Report, the "Reports on Domain Tasting"); Whereas, the GNSO Council has received the Draft Final Report on Domain Tasting; Whereas, PIR, the .org registry operator, has amended its Registry Agreement to charge an Excess Deletion Fee; and both NeuStar, the .biz registry operator, and Afilias, the .info registry operator, are seeking amendments to their respective Registry Agreements to modify the existing AGP; The GNSO Council recommends to the ICANN Board of Directors that: 1. The applicability of the Add Grace Period shall be restricted for any gTLD which has implemented an AGP ("Applicable gTLD Operator"). Specifically, for each Applicable gTLD Operator: a. During any given month, an Applicable gTLD Operator may not offer any refund to a registrar for any domain names deleted during the AGP that exceed (i) 10% of that registrar's net new registrations in that month (defined as total new registrations less domains deleted during AGP), or (ii) fifty (50) domain names, whichever is greater. b. A Registrar may seek an exemption from the application of such restriction in a specific month, upon the documented showing of extraordinary circumstances. For any Registrar requesting such an exemption, the Registrar must confirm in writing to the Registry Operator how, at the time the names were deleted, these extraordinary circumstances were not known, reasonably could not have been known, and were outside of the Registrar's control. Acceptance of any exemption will be at the sole reasonable discretion of the Registry Operator, however "extraordinary circumstances" which reoccur regularly will not be deemed extraordinary. c. In addition to all other reporting requirements to ICANN, each Applicable gTLD Operator shall identify each Registrar that has sought an exemption, along with a brief descriptive identification of the type of extraordinary circumstance and the action (if any) that was taken by the Applicable gTLD Operator. 2. Implementation and execution of these recommendations shall be monitored by the GNSO. Specifically; a. ICANN Staff shall analyze and report to the GNSO at six month intervals for two years after implementation, until such time as the GNSO resolves otherwise, with the goal of determining; i. How effectively and to what extent the policies have been implemented and followed by Registries and Registrars, and ii. Whether or not modifications to these policies should be considered by the GNSO as a result of the experiences gained during the implementation and monitoring stages, b. The purpose of these monitoring and reporting requirements are to allow the GNSO to determine when, if ever, these recommendations and any ensuing policy require additional clarification or attention based on the results of the reports prepared by ICANN Staff.

But there's this lovely new segment of lightly licensed spectrum in the U.S., the 3.65 GHz band. It's a non-exclusive licensed band available only in parts of the country that don't have pre-existing ground-to-satellite or radar uses that overlap. This omits most of the eastern seaboard and most major cities; Seattle is one exception.

The licensing mechanism allows any number of operators to obtain inexpensive licenses, and register the base stations they use by location. If interference arises among base stations, operators are required to work out the problems themselves. I wrote extensively about this band and its rules on 9-May-2008 in profiling Azulstar, formerly a metro-scale Wi-Fi firm, but now a big proponent of WiMax in 3.65 GHz. I also went over the rules for the band on 11-June-2007 when the FCC announced the arrangement.

Several firms offer base station and customer premises equipment for this band now, so close to the 3.5 GHz band more commonly exclusively licensed in Europe and elsewhere. WiMax equipment is available because the 3.65 GHz band can be used with WiMax without any modifications to that protocol, although limited to just 25 MHz of the 50 MHz that the FCC set aside.

Equipment that conforms to a more stringent set of rules about contention and other factors can use the whole 50 MHz, and that's where 802.11y comes in. It's an extension of Wi-Fi to cope with the specific needs--and to open Wi-Fi technology up to 20 W EIRP, a vastly higher power output. This could allow connections over 5 km, the group says.

The Wikipedia entry on 802.11y, clearly written by someone involved with the specification, notes that three specific additions are needed: a tweak to support the way in which the FCC wants contention among competing devices to work; a method for an access point to tell a station (a connecting radio) that it's about to switch its channel or its channel's bandwidth, and the station should do likewise; and a mechanism to handle a base station allowing or revoking permission to use the spectrum without uniquely identifying the user's system or broadcasting its precise GPS-based location.

The standard is near completion and initial approval. I don't have any knowledge about whether any mainstream Wi-Fi equipment makers or metro-scale equipment makers are looking into building 802.11y into their gear.

The fact is that this could be a great technology for the mostly sub-metropolitan markets that 3.65 GHz is available in, although it has the same pain as WiMax: all new gear on the towers and all new adapters for customers.

First it was the EU looking at passing a law that would require bloggers to disclose their identity and affiliation. Now the AP is looking to enforce a new license that would require payments when a blogger puts an excerpt from an AP article in their blog.  My friend Kevin McLaughlin blogged on this over at Channel Web blog today. Basically the AP says that if you excerpt more than 5 words you need to start paying them fees.  Kevin reached out to me and I gave him my views on this one.

I think that it is a really short sighted move by the AP.  First of all it shows they really don't understand blogging.  Blogging is about taking an idea which often comes from another source and putting the bloggers own spin and ideas behind it. In this way topics are built on one blog at a time with each blogger adding a bit more to the conversation. Each additional blog on topic enriches those blogs and articles that preceded it.  As I said in the Channel Web article, it is like a jazz musician playing a riff on top of a line already laid down.

In real terms blogging on the AP content will only generate more views and interest in the AP content.  AP is just a dinosaur with this type of view and will soon go the way of dinosaurs if they try to enforce this. In the meantime bloggers can talk about an AP article, but don't link to it and don't excerpt from it. I suspect that the next thing is we will have a replay of the inbound links litigation we had 8 years ago.  In the meantime blogging will continue to march on with AP or not.