The new index sheds much needed light on the response time and success rate for performing typical transactions on leading federal government Web sites. For the first week of official results index performance varied dramatically with response times between 1.33 seconds (FDA Web site) and 16.88 seconds (Department of Transportation Web site) and success rates between 99.91 percent availability (IRS and SEC Web sites) and 91.08 percent availability (Department of Transportation Web site).
"Response time" is defined as the aggregate time in seconds it took during the week to complete a multi-step transaction from the 10 U.S. cities. A transaction consists of four components (that may represent four or more pages in an e-government transaction).
"Success rate" is defined as the aggregate number times out of 100 that Keynote's measurement computers are able to complete the entire scripted transaction from start to finish.
The top five index performers will be published weekly on Keynote's Web site at: www.keynote.com/solutions/performance_indices/government_txn_index/gov_txn_index.html
The URL to the index on Keynote's Web site will link to the top 5 performers on the index the previous week and the worst performer anonymously, while the full results of the weekly government transaction index (all 12 sites) will appear exclusively on the Web site of and in the pages of Government Computer News (NYSE: WPO), the leading technology magazine for government. Click here for the complete results for the week of February 9: http://gcn.com/Resource/itinfrastructure/Web_Performance.html
"If the government truly wants to provide better service online, then consistent and fast Web site performance is crucial. This index will let GCN readers take regular measure of their own online health," said Thomas R. Temin, editor-in-chief of Government Computer News. "We're proud to work with Keynote to get this index before people who can do something about federal site performance."
To illustrate the importance of the Internet to U.S. citizens and the high expectations they have, a recent survey of 1,023 adults, conducted by Hart-Teeter Research and underwritten by Accenture, showed that 74 percent of e-government users expect that the Internet will have a more positive effect on how government operates over the next 5 - 10 years. A separate study by Accenture found that the number one reason governments use the Internet is to improve customer service. Therefore, it stands to reason that assuring optimal e-government performance from the citizen's perspective (the goal for the launch of Keynote's new e-government transaction index) is one of the most critical components of every federal agency's Web mandate.
Keynote has historically seen an improvement in overall performance results, over time, whenever it has launched a new vertical-based Web transaction performance index; in this case the government sector. By increasing public awareness for the transaction performance of leading government Web sites, Keynote creates a benchmark for Web performance that government agencies (city, state and federal) can strive to reach.
The new Keynote government transaction index gives government agencies a very useful and valuable tool to compare, contrast and improve their overall performance against both internal metrics and goals set for performance, as well as against one another. The overall goal for the index is to help e-government measure and manage the performance of their Web sites and optimize the experience citizens receive when searching for information or doing business with the government online.
"Shedding light on the performance of leading government Web sites supports the federal government's online initiative to allow citizens to find services, do business and connect with their government with the click of a computer mouse. The new index helps fulfill that admirable goal," said Roopak Patel, senior Internet analyst in Keynote's public services division. "Currently, there is a surprisingly wide range of disparity in performance, some of the largest we've seen. Over time, we expect to see this disparity narrow, as the sites strive to bring themselves more in line with the performance expectations consumers have come to expect from commercially-based Web sites."
The Keynote E-Government Web Transaction Performance Index is designed to reflect on the speed and success rates for the top, most commonly accessed government related Web sites. Going beyond the home page performance, this index provides deeper insight into the performance of these high traffic Web sites. The methodology consists of three simple steps: accessing the home page, entering search criteria and selecting or clicking on a specific returned result.
Here is the summary data for the first week's worth of results from Keynote's new E-Government Web Transaction Performance Index:
Week of February 9, 2004, 5:00 AM to 9:00 PM Pacific Time: RESPONSE TIME SUCCESS RATE ---------------------------------- ----------------------------------- Rank Site Time Rank Site Success Rate (seconds) (percent) ---- -------------------- ------- ---- --------------------- ------- 1 FDA 1.33 1 IRS 99.91 2 Homeland Security 1.56 2 SEC 99.91 3 SEC 2.13 3 FirstGov 99.82 4 U.S. House of Rep. 2.23 4 FDA 99.73 5 FirstGov 3.40 5 NASA 99.64 6 FAA 3.56 6 U.S. House of Reps. 99.64 KEYNOTE INDEX 3.92 7 Dept. of Vet. Affairs 99.20 7 NASA 5.56 8 NIH 98.84 8 NIH 6.41 9 FAA 98.36 9 Dept. of Education 6.49 KEYNOTE INDEX 98.23 10 IRS 7.69 10 Dept. of Educ. 97.84 11 Dept. of Vet Affairs 10.52 11 Dept. Home. Sec. 94.70 12 Dept. of Transp. 16.88 12 Dept. of Transp. 91.08 ----------------------------------------------------------------------The twelve U.S. federal government Web sites included on Keynote's e-government transaction performance index are:
- Internal Revenue Service: www.irs.gov
- Food and Drug Administration: www.fda.gov
- Homeland Security: www.ready.gov
- National Institutes of Health: www.nih.gov
- NASA: www.nasa.gov/home/index.html
- Securities and Exchange Commission: www.sec.gov
- Veterans Affairs: www.va.gov
- FirstGov: www.firstgov.gov
- Department of Education: www.ed.gov
- Department of Transportation: www.dot.gov
- U.S. House of Representatives: www.house.gov
- Federal Aviation Administration www.faa.gov
The data used to produce the index for a given week is taken from actual online transactions Keynote's automated measurement computers execute and measure from the 10 U.S. cities (Boston, San Francisco, Detroit, Washington, D.C., Los Angeles, New York, Chicago, Philadelphia, Houston and Dallas) on an hourly basis Monday to Sunday from 5:00 a.m. to 9:00 p.m. pacific time.
About Keynote's E-Government Web Transaction Performance Index
Keynote pioneered the use of vertical indices and benchmarks for Web application performance over five years ago.
The new Keynote E-Government Web Transaction Performance Index is the federal government's most useful and accurate Web transaction performance benchmarking index for improving the online performance of government Web sites. Keynote measures e-government transactions from all the major metropolitan areas in the United States and places its transaction measurement computers on the most widely used, familiar and recognizable Internet backbone and service providers in the United States including Uunet/Worldcom, Sprint, AT&T and Qwest.
Additionally, Keynote's global network of measurement computers run standard Microsoft Internet Explorer browsers on Windows/2000 to obtain the most accurate and representative transaction performance data possible today.
For the first time government Web site operators have access to a useful and cost-effective benchmarking tool that provides real time, 24/7 visibility into the performance of their sites. The e-government transaction performance index allows government agencies to meet internal Web performance objectives and to compare and contrast their online performance to other government agencies.
Complete index data that includes not only the aggregated and limited time-frame results as published here, but every data point taken 24/7 along with page, network component, content and error detail, is available from Keynote on a subscription basis for $3,885.00 per month. An aggregate view of the data is available for $895.00 per month.
Background on Keynote's Web Performance Indices
Keynote's Web transaction indices leverage Keynote's long history and experience developing Internet performance indices. Keynote launched the world's first Web performance indices that assess page download performance (as opposed to transaction performance), known as The Keynote Business 40 Internet Performance Index (measurements over T1 and T3 connections) and The Keynote Consumer 40 Internet Performance Index (measurements over dial-up and broadband connections). Keynote also launched the world's first transaction performance index, The Keynote Broker Transaction Index. All today are considered the 'gold standard' for Web performance benchmarking.
Now customers can use Keynote's growing line of industry-specific Web transaction indices to assess Web transaction performance against internal benchmarks and against competitors. Customers can then leverage Keynote's additional monitoring, performance management, load, content and usability testing services to improve the overall performance of their e-business applications and infrastructure.
To build and maintain its growing line of Web performance indices, Keynote uses its unparalleled global infrastructure of over 1,700 strategically and statistically located measurement, testing and diagnostic computers representing Internet performance from 50 metropolitan areas worldwide. Information about all Keynote performance indices can be found at www.keynote.com/solutions/solutions_pm_performance_indices_tpl.html
About Keynote
Founded in 1995, Keynote Systems, Inc., (Nasdaq "KEYN"), The Internet Performance Authority(R), is the worldwide leader in Web performance measurement and management services that improve the quality of e-business. Keynote's services enable corporate enterprises to monitor, benchmark, test, diagnose and optimize their e-business systems both inside and outside the firewall. Approximately 2,200 corporate IT departments and 18,000 individual subscribers rely on the company's easy-to-use and cost-effective services to increase revenues and reduce downtime costs, without requiring additional complex and costly software implementations.
Keynote Systems, Inc. is headquartered in San Mateo, California and can be reached at www.keynote.comor by phone in the U.S. at 650-403-2400.
Keynote, The Internet Performance Authority and Perspective are registered trademarks of Keynote Systems, Inc. Other trademarks are the property of their respective owners.
