IMS/ISC: Comments on .org Preliminary Evaluation Report.
The analysis was an impressive display of committee logistics and resulted in three different rating algorithms, which fed a staff meta-algorithm, which in turn produced a summary result. Unfortunately, your code didn’t compile on our systems:
- There was no technical due diligence conducted as part of the evaluation. Past performance claims are not documented and no examination of code, logs, or configurations was conducted.
- The evaluation was all theory and no practice. Since the distance between theory and practice is so much smaller in theory than in practice, it is no surprise that the evaluations were able to boil complex problems down to simple metrics.
ICANNWatch: Old Internet Thinking RIP. “See, the problem is that Malamud’s entire essay is consumed with irrelevant Old Internet considerations like running code, technical merit, and whether it makes sense to evaluate a program without ever looking at it. This IETF-style approach to the problem of finding reasonable solutions to problems has no place in the Brave New Internet of today where expensive consulting firms decide that proposals produced by expensive consulting firms have the most merit, where merit is defined as producing familiar-looking paper.”
SWITCH Comments: “This evaluation process was an All-American play.”
DotOrg Foundation Comments: “Our spreadsheets demonstrate what happens when one or more errors are corrected. Scoring can shift to a startling degree.”
.Org Foundation Comments: “The NCDNHC Report stated on page 20 that The .Org Foundation “did not respond to the NCDNHC questions, nor to any other substantive questions on the list”. This is a completely false and highly prejudicial statement.” “Based on the significant inaccuracies stated as justification for our low ranking we hereby request that Gartner or the ICANN Staff re-evaluate our proposal and publish the findings.”
Organic Names Comments: “Emails that appeared to be from the subcommittees (whose identities were not known to Organic Names) were referred to ICANN to determine whether ICANN wished us to answer them. In each case, the answer was ‘no’.”