- The US Department of Homeland Security (DHS) has been working in secret for more than two weeks with the private sector to fix a major internet vulnerability that could have had disastrous consequences for millions of businesses and the US military.
Since February 14, the DHS and the White House Office of Cyberspace Security have been working with Atlanta-based Internet Security Systems (ISS) to alert IT vendors and the business community about a major buffer overflow vulnerability in the sendmail mail transfer agent (MTA).
Sendmail is the most common MTA and handles between 50% and 75% of all internet email traffic. Versions of the software, from 5.79 to 8.12.7, are vulnerable, according to an ISS alert issued publicly Monday (US time).
According to sources familiar with the investigation, ISS discovered the vulnerability on February 13. It then contacted the homeland security officials, who began the process of alerting IT vendors that distribute sendmail, including Sun Microsystems, IBM, Hewlett-Packard and Silicon Graphics, as well as the Sendmail Consortium, the organisation that develops the open-source version of sendmail that is distributed with both free and commercial operating systems. The seriousness of the vulnerability, coupled with the fact that the hacker community wasn't yet aware of it, caused the government and ISS to decide it was better to keep the news under wraps until patches could be developed.
The Sendmail Consortium is urging all users to either upgrade to Sendmail 8.12.8 or apply a patch for 8.12.x (or for older versions). Updates can be downloaded from ftp.sendmail.org or any of its mirrors, or from the Sendmail Consortium's site. The consortium says patch users should remember to check the PGP signatures of any patches or releases obtained. It also suggested that those running the open-source version of sendmail check with their vendors for a patch.
Sendmail, the commercial provider of the sendmail MTA, is providing a binary patch for its commercial customers that can be downloaded from Sendmail's website.
"The Remote Sendmail Header Processing Vulnerability allows local and remote users to gain almost complete control of a vulnerable Sendmail server," according to an alert prepared Monday by the DHS. "Attackers gain the ability to execute privileged commands using super-user (root) access/control. This vulnerability can be exploited through a simple email message containing malicious code.
"System administrators should be aware that many Sendmail servers are not typically shielded by perimeter defense applications" such as firewalls, warned the DHS alert, which hadn't yet been made publicly available as of mid-afternoon. "A successful attacker could install malicious code, run destructive programs and modify or delete files."
Additionally, attackers could gain access to other systems through a compromised sendmail server, depending on local configurations, according to the DHS warning.
According to the ISS, the sendmail remote vulnerability occurs when processing and evaluating header fields in email collected during a Simple Mail Transfer Protocol transaction. Specifically, when fields are encountered that contain addresses or lists of addresses (such as the "From" field, "To" field and "CC" field), sendmail attempts to semantically evaluate whether the supplied address (or list of addresses) is valid. This is accomplished using the crackaddr() function, which is located in the headers.c file in the Sendmail source tree.
A static buffer is used to store data that has been processed. Sendmail detects when this buffer becomes full and stops adding characters, although it continues processing. Sendmail implements several security checks to ensure that characters are parsed correctly. One such security check is flawed, making it possible for a remote attacker to send an e-mail with a specially crafted address field that triggers a buffer overflow.
"Sendmail's vulnerability offers a legitimate test [of the new DHS and its ability to work with the private sector] because sendmail handles a large amount of internet mail traffic and is installed on at least 1.5 million internet-connected systems," said an alert from the SANS Institute in Bethesda, Maryland, that was obtained by Computerworld US on Monday.
"More than half of the large ISPs and Fortune 500 companies use sendmail, as do tens of thousands of other organisations. A security hole in sendmail affects a lot of people and demands their immediate attention."
Of particular concern to the White House was the potential vulnerability of the US military, which is poised to begin offensive military operations in Iraq and is simultaneously facing the possibility of conflict on the Korean peninsula. As a result, early versions of available patches were distributed first to US military organizations on February 25 and 26, according to the SANS alert. The advance military alert was followed last Thursday and Friday with alerts to various government organisations in the US and around the world, including the Information Sharing and Analysis Centers (ISAC).
"Some of the large commercial vendors developed patches very quickly. But the delayed notice to smaller sources of sendmail distributions and limited resources at those organizsations meant that not all the patches would be ready by early in the week of February 23," according to the SANS analysis of the public-private response effort.
A senior-level coordination group of government and private-sector experts then decided, based on a review of cyber intelligence from various hacker discussion boards and a series of sensors deployed around the world by ISS, that it was safe to wait until all the patches were available before alerting the general business and internet community to the vulnerability.
Beginning Monday at 10am EST, alerts began flowing to federal agencies from the Federal Computer Incident Response Center (FedCIRC) and, from the ISACs, to companies responsible for critical infrastructure. At noon EST today, ISS released its own advisory, followed by a general alert from the CERT Coordination Center at Carnegie Mellon University in Pittsburgh.