Yo Yo! You must login or signup first!

Linux_hack

Submission   5,253

Part of a series on Linux. [View Related Entries]


About

XZ Backdoor Linux Hack refers to the discovery of a malicious "backdoor" in XZ compression tools used in some newer versions of open-source Linux systems. Microsoft engineer Andres Freund found the backdoor in late March 2024 after he noticed a half-second lag while logging into his machine. Further investigation revealed code introduced into the system over the course of two years by a developer named Jia Tan that aimed to grant remote access to every computer running the latest versions of Linux-based operating systems Ubuntu, Fedora and Debian. Engineers and tech enthusiasts shared memes about Freund's discovery, with many jokes leveraging the fact that he was tipped off by a mere 500-millisecond delay in his system.

Origin

On March 29th, 2024, Microsoft engineer Andres Freund made a post on Mastodon[1] urging people who run "Debian Testing, Unstable, or some other more 'bleeding edge' distribution" to upgrade their Linux systems as soon as possible (seen below). He shared a document about how he discovered a backdoor in the XZ compression tool used in Linux distributions, saying that he was tipped off by a minute lag in his system.

AndresFreundTec @AndresFreund Tec I accidentally found a security issue while benchmarking postgres changes. If you run debian testing, unstable or some other more "bleeding edge" distribution, I strongly recommend upgrading ASAP. openwall.com/lists/oss-securit... ← 94 www.openwall.com oss-security-backdoor in upstream xz/l... = 1d ••• AndresFreundTec @AndresFreundTec@mastodon.social I was doing some micro-benchmarking at the time, needed to quiesce the system to reduce noise. Saw sshd processes were using a surprising amount of CPU, despite immediately failing because of wrong usernames etc. Profiled sshd, showing lots of cpu time in liblzma, with perf unable to attribute it to a symbol. Got suspicious. Recalled that I had seen an odd valgrind complaint in automated testing of postgres, a few weeks earlier, after package updates. Really required a lot of coincidences. Mar 29, 2024, 02:32 PM • Web

The backdoor commits were allegedly added by a developer known online as Jia Tan over the course of two years, with work logs indicating that their contributions took place at regular weekly intervals as part of a 9-5 job (seen below, left).[3] Moreover, some developers shared on YCombinator[4] that Jia Tan had urged them to add the compromised XZ feature to newer versions of Fedora (seen below, right).

hackerfantastic.x @hackerfantastic Follow The author of the 'xz' backdoor commit history and activity shows that they kept office hours mostly. Mon-Fri, every other Saturday, I would imagine some of these would correlate with public holidays as this was clearly not a hobbyist. github.com/ JiaT75?tab=ove... 413 contributions in 2023 Mon Wed Fri Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Learn how we count contributions 8:06 PM - Mar 29, 2024 781K Views Less More
rwmj 1 day ago | next [-] Very annoying - the apparent author of the backdoor was in communication with me over several weeks trying to get xz 5.6.x added to Fedora 40 & 41 because of it's "great new features". We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor he had added). We had to race last night to fix the problem after an inadvertent break of the embargo. He has been part of the xz project for 2 years, adding all sorts of binary test files, and to be honest with this level of sophistication I would be suspicious of even older versions of xz until proven otherwise. reply

Spread

News about Andres Freund's discovery made it to X / Twitter on March 29th, 2024, with X[2][5] user @thegrugq questioning the role of Jia Tan in the hack, saying that it is unlikely that they were an innocent but compromised developer, and adding that the end goal of the hack would have been access to every system running Fedora, Debian and Ubuntu. The post gathered over 5,000 likes in a day (seen below).

• thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 On the .xz backdoor. It is hard to see how the developer Jia Tan is innocent. The backdoor was added in 5.6.0 by his account. He contacted Fedora to push them to move to 5.6.0. There was a problem with valgrind, they worked with hi to resolve it. He commits the fix in 5.6.1. 33 17675 4.9K Il 953K • thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 If Jia Tan did not commit the backdoor in 5.6.0, and his account was hijacked, it strains credulity that he worked on fixing an issue introduced from a fraudulent commit in his name without noticing. Instead, he worked with Fedora to resolve the issue and committed a fix. 3 17 16 580 Ill 58K thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 Fedora: news.ycombinator.com/item?id=398658... OSS announce: openwall.com/lists/oss-secu... Backdoor Commit: tukaani-project/xz Tests: Add a few test files. 19 lines changed +19-0 JiaT75 committed February 23, 2024 --cf44e4b Tests: Add a few test files. • tukaani-project/xz@cf44e4b From github.com 3 27 17 315 IlII 61K
thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 On the .xz backdoor. It is hard to see how the developer Jia Tan is innocent. The backdoor was added in 5.6.0 by his account. He contacted Fedora to push them to move to 5.6.0. There was a problem with valgrind, they worked with hi to resolve it. He commits the fix in 5.6.1. 33 17 675 Jacob S @ResidentMemer Mar 29 • 4.9K Il 952K ↑ Let's call it. What's the intended malicious end to this? What is the purpose of this backdoor? Q1 271 180 115K thaddeus e. grugq thegrugq@infosec.exchange @thegrugq. Mar 29 According to the reports it provides remote access. There appears to be a method for a malicious user with an rsa key to authenticate to the backdoored sshd. ○ 2 17 14 604 Il 116K Jacob S @ResidentMemer · Mar 29 That's the functionality, what is the end game? It seems like more than just a cash-grab/ransom operation. Am I cynical to suspect a state actor? 3 173 204 III 103K ☐ 1 thaddeus e. grugq thegrugq@infosec.exchange @thegrugq The end game would be the ability to login to every Fedora, Debian and Ubuntu box on the internet. If it isn't a state actor it should be... 6:57 PM Mar 29, 2024 1.7M Views

On March 30th, Redditors[9] on /r/Linux theorized that the XZ backdoor must have been the work of malicious state actors, with Jia Tan being a pseudonym for an entire team of workers undertaking a long-term state-sponsored campaign (seen below. left). Also on March 30th, Redditor[10] /u/shy_cthulhu posted a Cat Looks Inside meme about he backdoor to /r/linuxmemes, gathering over 800 upvotes in a day.

[-] Mysterious_Focus6144 352 points 17 hours ago The hooked RSA_public_decrypt verifies a signature on the server's host key by a fixed Ed448 key, and then passes a payload to system(). It sounds like the backdoor attempt was meant as the first step of a larger campaign: 1. Create backdoor. 2. Remotely execute an exploit. 3. profit. This methodical, patient, sneaky effort spanning a couple of years makes it more likely, to me at least, to be the work of a state, which also seems to be the consensus atm permalink source embed save save-RES report reply hide child comments [-] ProgsRS 159 points 16 hours ago It's very likely to be a planned group project given the amount of time it took. Less likely for a lone actor to have this much patience, foresight and commitment. There were others involved as fresh accounts who played different roles (like pressuring the maintainer) during certain periods and suddenly dropped off after, while Jia Tan was a separate persona who had been slowly and separately building trust with the end goal and task of delivering the final payload. It's possible that this was all the same person switching roles, but it's more likely to be an organized group effort over the span of years. permalink source embed save save-RES parent report reply hide child comments
>Performance testing regressions >Look inside >They put a backdoor in xz

On March 30th, 2024, X[6] user @vxunderground made a post calling Andres Freund the "silver back gorilla of nerds" and the "internet final boss," gathering over 18,000 likes in a day (seen below, left). That same day, X[7] user @0x_shaq joked about installing the compromised software on their system, gathering over 3,000 likes in a day (seen below, right).

UNDERGROU vx-underground @vxunderground : The xz backdoor was initially caught by a software engineer at Microsoft. He noticed 500ms lag and thought something was suspicious. This is the Silver Back Gorilla of nerds. The internet final boss. dTec undTec@mastodon.social e micro-benchmarking at the time, em to reduce noise. Saw sshd prod g amount of CPU, despite immedia g usernames etc. Profiled sshd, sh na, with perf unable to attribute it t alled that I had seen an odd valgrin zma installed, logins via ssh beco alhost ermission denied (publickey). ng of postgres, a few weeks earlier, ermission denied (publickey). lot of coincidences. 9:49 AM · Mar 30, 2024 3.7M Views
faulty *ptrrr @0x_shaq Finally I found the time to update some packages on my laptop. I feel so proud. ==> Upgrading 1 outdated package: xz 5.5.0 -> 5.6.0 ... Downloading https://ghcr.io/v2 Fetching xz Downloading https://ghcr.io/v2 1:02 PM Mar 30, 2024 108.6K Views

Also on March 30th, X[8] user @mippl3 posted a video of Steph Curry discovering a defect in the basketball court because of a failed dribble, writing, "This is explains how the xz backdoor was found," and gathering over 12,000 likes in a day (seen below).

Various Examples

XZ Utils Backdoor It seems to me that this backdoor got everyone worried for the wrong reasons... And what would be the right reasons? NY. 2024 Patch Friday Backdoors without performance issues.
BASED OFF COMMIT TIMES WE BELIEVE THIS TO BE A CHINA-NEXUS STATE SPONSORED THREAT ACTOR maflin.com
Ο Xz xz backdoor is patched. PATCH Amazing! 1 ΠΠΠΕ Daniel Stori {turnoffius}
PROPRIETARY BACKDOOR MENTAL GYMNASTICS I want a backdoor in this software I ask the corpo to grant me access OPEN SOURCE BACKDOOR MENTAL GYMNASTICS spend 5 years gaining Immediately get find not well maintained related project the mantainer trust noticed by an Insert brittle system to autist due to 0.5s startup delay introduce vulnerability
15M+LAG IN MS TEAMS 500MS LAG IN LIBLZMA

Search Interest

Unavailable.

External References

[1] Mastodon – AndresFreundTec

[2] X – thegrugq

[3] X – hackerfantastic

[4]  YCombinator – Backdoor

[5] X – thegrugq

[6] X – vxunderground

[7] X – 0x_shaq

[8] X – mippl3

[9] Reddit – /r/linux

[10] Reddit – /r/linuxmemes



Share Pin

Related Entries 3 total

1336193660379
Install Gentoo
Tco_-_2021-11-09t131430.682
BTW I Use Arch
Xenia_(linuxfox)
Xenia (LinuxFox)

Recent Images 20 total


Recent Videos 0 total

There are no recent videos.




Load 7 Comments
XZ Backdoor Linux Hack image example.

XZ Backdoor Linux Hack

Part of a series on Linux. [View Related Entries]

Updated Apr 01, 2024 at 01:57PM EDT by Zach.

Added Mar 31, 2024 at 05:45AM EDT by sakshi.

PROTIP: Press 'i' to view the image gallery, 'v' to view the video gallery, or 'r' to view a random entry.

This submission is currently being researched & evaluated!

You can help confirm this entry by contributing facts, media, and other evidence of notability and mutation.

About

XZ Backdoor Linux Hack refers to the discovery of a malicious "backdoor" in XZ compression tools used in some newer versions of open-source Linux systems. Microsoft engineer Andres Freund found the backdoor in late March 2024 after he noticed a half-second lag while logging into his machine. Further investigation revealed code introduced into the system over the course of two years by a developer named Jia Tan that aimed to grant remote access to every computer running the latest versions of Linux-based operating systems Ubuntu, Fedora and Debian. Engineers and tech enthusiasts shared memes about Freund's discovery, with many jokes leveraging the fact that he was tipped off by a mere 500-millisecond delay in his system.

Origin

On March 29th, 2024, Microsoft engineer Andres Freund made a post on Mastodon[1] urging people who run "Debian Testing, Unstable, or some other more 'bleeding edge' distribution" to upgrade their Linux systems as soon as possible (seen below). He shared a document about how he discovered a backdoor in the XZ compression tool used in Linux distributions, saying that he was tipped off by a minute lag in his system.


AndresFreundTec @AndresFreund Tec I accidentally found a security issue while benchmarking postgres changes. If you run debian testing, unstable or some other more "bleeding edge" distribution, I strongly recommend upgrading ASAP. openwall.com/lists/oss-securit... ← 94 www.openwall.com oss-security-backdoor in upstream xz/l... = 1d ••• AndresFreundTec @AndresFreundTec@mastodon.social I was doing some micro-benchmarking at the time, needed to quiesce the system to reduce noise. Saw sshd processes were using a surprising amount of CPU, despite immediately failing because of wrong usernames etc. Profiled sshd, showing lots of cpu time in liblzma, with perf unable to attribute it to a symbol. Got suspicious. Recalled that I had seen an odd valgrind complaint in automated testing of postgres, a few weeks earlier, after package updates. Really required a lot of coincidences. Mar 29, 2024, 02:32 PM • Web

The backdoor commits were allegedly added by a developer known online as Jia Tan over the course of two years, with work logs indicating that their contributions took place at regular weekly intervals as part of a 9-5 job (seen below, left).[3] Moreover, some developers shared on YCombinator[4] that Jia Tan had urged them to add the compromised XZ feature to newer versions of Fedora (seen below, right).


hackerfantastic.x @hackerfantastic Follow The author of the 'xz' backdoor commit history and activity shows that they kept office hours mostly. Mon-Fri, every other Saturday, I would imagine some of these would correlate with public holidays as this was clearly not a hobbyist. github.com/ JiaT75?tab=ove... 413 contributions in 2023 Mon Wed Fri Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Learn how we count contributions 8:06 PM - Mar 29, 2024 781K Views Less More rwmj 1 day ago | next [-] Very annoying - the apparent author of the backdoor was in communication with me over several weeks trying to get xz 5.6.x added to Fedora 40 & 41 because of it's "great new features". We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor he had added). We had to race last night to fix the problem after an inadvertent break of the embargo. He has been part of the xz project for 2 years, adding all sorts of binary test files, and to be honest with this level of sophistication I would be suspicious of even older versions of xz until proven otherwise. reply

Spread

News about Andres Freund's discovery made it to X / Twitter on March 29th, 2024, with X[2][5] user @thegrugq questioning the role of Jia Tan in the hack, saying that it is unlikely that they were an innocent but compromised developer, and adding that the end goal of the hack would have been access to every system running Fedora, Debian and Ubuntu. The post gathered over 5,000 likes in a day (seen below).


• thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 On the .xz backdoor. It is hard to see how the developer Jia Tan is innocent. The backdoor was added in 5.6.0 by his account. He contacted Fedora to push them to move to 5.6.0. There was a problem with valgrind, they worked with hi to resolve it. He commits the fix in 5.6.1. 33 17675 4.9K Il 953K • thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 If Jia Tan did not commit the backdoor in 5.6.0, and his account was hijacked, it strains credulity that he worked on fixing an issue introduced from a fraudulent commit in his name without noticing. Instead, he worked with Fedora to resolve the issue and committed a fix. 3 17 16 580 Ill 58K thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 Fedora: news.ycombinator.com/item?id=398658... OSS announce: openwall.com/lists/oss-secu... Backdoor Commit: tukaani-project/xz Tests: Add a few test files. 19 lines changed +19-0 JiaT75 committed February 23, 2024 --cf44e4b Tests: Add a few test files. • tukaani-project/xz@cf44e4b From github.com 3 27 17 315 IlII 61K thaddeus e. grugq thegrugq@infosec.exchange @thegrugq Mar 29 On the .xz backdoor. It is hard to see how the developer Jia Tan is innocent. The backdoor was added in 5.6.0 by his account. He contacted Fedora to push them to move to 5.6.0. There was a problem with valgrind, they worked with hi to resolve it. He commits the fix in 5.6.1. 33 17 675 Jacob S @ResidentMemer Mar 29 • 4.9K Il 952K ↑ Let's call it. What's the intended malicious end to this? What is the purpose of this backdoor? Q1 271 180 115K thaddeus e. grugq thegrugq@infosec.exchange @thegrugq. Mar 29 According to the reports it provides remote access. There appears to be a method for a malicious user with an rsa key to authenticate to the backdoored sshd. ○ 2 17 14 604 Il 116K Jacob S @ResidentMemer · Mar 29 That's the functionality, what is the end game? It seems like more than just a cash-grab/ransom operation. Am I cynical to suspect a state actor? 3 173 204 III 103K ☐ 1 thaddeus e. grugq thegrugq@infosec.exchange @thegrugq The end game would be the ability to login to every Fedora, Debian and Ubuntu box on the internet. If it isn't a state actor it should be... 6:57 PM Mar 29, 2024 1.7M Views

On March 30th, Redditors[9] on /r/Linux theorized that the XZ backdoor must have been the work of malicious state actors, with Jia Tan being a pseudonym for an entire team of workers undertaking a long-term state-sponsored campaign (seen below. left). Also on March 30th, Redditor[10] /u/shy_cthulhu posted a Cat Looks Inside meme about he backdoor to /r/linuxmemes, gathering over 800 upvotes in a day.


[-] Mysterious_Focus6144 352 points 17 hours ago The hooked RSA_public_decrypt verifies a signature on the server's host key by a fixed Ed448 key, and then passes a payload to system(). It sounds like the backdoor attempt was meant as the first step of a larger campaign: 1. Create backdoor. 2. Remotely execute an exploit. 3. profit. This methodical, patient, sneaky effort spanning a couple of years makes it more likely, to me at least, to be the work of a state, which also seems to be the consensus atm permalink source embed save save-RES report reply hide child comments [-] ProgsRS 159 points 16 hours ago It's very likely to be a planned group project given the amount of time it took. Less likely for a lone actor to have this much patience, foresight and commitment. There were others involved as fresh accounts who played different roles (like pressuring the maintainer) during certain periods and suddenly dropped off after, while Jia Tan was a separate persona who had been slowly and separately building trust with the end goal and task of delivering the final payload. It's possible that this was all the same person switching roles, but it's more likely to be an organized group effort over the span of years. permalink source embed save save-RES parent report reply hide child comments >Performance testing regressions >Look inside >They put a backdoor in xz

On March 30th, 2024, X[6] user @vxunderground made a post calling Andres Freund the "silver back gorilla of nerds" and the "internet final boss," gathering over 18,000 likes in a day (seen below, left). That same day, X[7] user @0x_shaq joked about installing the compromised software on their system, gathering over 3,000 likes in a day (seen below, right).


UNDERGROU vx-underground @vxunderground : The xz backdoor was initially caught by a software engineer at Microsoft. He noticed 500ms lag and thought something was suspicious. This is the Silver Back Gorilla of nerds. The internet final boss. dTec undTec@mastodon.social e micro-benchmarking at the time, em to reduce noise. Saw sshd prod g amount of CPU, despite immedia g usernames etc. Profiled sshd, sh na, with perf unable to attribute it t alled that I had seen an odd valgrin zma installed, logins via ssh beco alhost ermission denied (publickey). ng of postgres, a few weeks earlier, ermission denied (publickey). lot of coincidences. 9:49 AM · Mar 30, 2024 3.7M Views faulty *ptrrr @0x_shaq Finally I found the time to update some packages on my laptop. I feel so proud. ==> Upgrading 1 outdated package: xz 5.5.0 -> 5.6.0 ... Downloading https://ghcr.io/v2 Fetching xz Downloading https://ghcr.io/v2 1:02 PM Mar 30, 2024 108.6K Views

Also on March 30th, X[8] user @mippl3 posted a video of Steph Curry discovering a defect in the basketball court because of a failed dribble, writing, "This is explains how the xz backdoor was found," and gathering over 12,000 likes in a day (seen below).


Various Examples


XZ Utils Backdoor It seems to me that this backdoor got everyone worried for the wrong reasons... And what would be the right reasons? NY. 2024 Patch Friday Backdoors without performance issues. BASED OFF COMMIT TIMES WE BELIEVE THIS TO BE A CHINA-NEXUS STATE SPONSORED THREAT ACTOR maflin.com Ο Xz xz backdoor is patched. PATCH Amazing! 1 ΠΠΠΕ Daniel Stori {turnoffius} PROPRIETARY BACKDOOR MENTAL GYMNASTICS I want a backdoor in this software I ask the corpo to grant me access OPEN SOURCE BACKDOOR MENTAL GYMNASTICS spend 5 years gaining Immediately get find not well maintained related project the mantainer trust noticed by an Insert brittle system to autist due to 0.5s startup delay introduce vulnerability 15M+LAG IN MS TEAMS 500MS LAG IN LIBLZMA

Search Interest

Unavailable.

External References

[1] Mastodon – AndresFreundTec

[2] X – thegrugq

[3] X – hackerfantastic

[4]  YCombinator – Backdoor

[5] X – thegrugq

[6] X – vxunderground

[7] X – 0x_shaq

[8] X – mippl3

[9] Reddit – /r/linux

[10] Reddit – /r/linuxmemes

Recent Videos

There are no videos currently available.

Recent Images 20 total


Top Comments


+ Add a Comment

Comments (7)


Display Comments

Add a Comment