:The most recent “security advisory” was released despite the fact
: that the particular bug in the experimental HTTP/3 code is
: expected to be fixed as a normal bug as per the existing security
: policy, and all the developers, including me, agree on this.
:
: And, while the particular action isn’t exactly very bad, the
: approach in general is quite problematic.
There was no public discussion. The only discussion I’m aware of
happened on the security-alert@ list, and the consensus was that
the bug should be fixed as a normal bug. Still, I was reached
several days ago with the information that some unnamed management
requested an advisory and security release anyway, regardless of
the policy and developers position.
Historically, we did not issue CVEs for experimental features and instead would patch the relevant code and release it as part of a standard release. For commercial customers of NGINX Plus, the previous two versions would be patched and released to customers. We felt that not issuing a similar patch for NGINX Open Source would be a disservice to our community. Additionally, fixing the issue in the open source branch would have exposed users to the vulnerability without providing a binary.
Our decision to release a patch for both NGINX Open Source and NGINX Plus is rooted in doing what is right – to deliver highly secure software for our customers and community. Furthermore, we’re making a commitment to document and release a clear policy for how future security vulnerabilities will be addressed in a timely and transparent manner.
I…agree with the “company” I think. This sounds like dev sour grapes but what the company was asking them to do seems better from the customer pov and for cyber security I’m general.
This sounds like dev sour grapes but what the company was asking them to do seems better from the customer pov and for cyber security I’m general.
As a developer myself (though not on the level of these guys): sorry, but just, no.
The key point is this:
[…] we did not issue CVEs for experimental features and instead would patch the relevant code and release it as part of a standard release.
Emphasis mine. In software, features marked as “experimental” usually are not meant to be used in a production environment, and if they are, it’s in a “do it at your own risk” understanding. Software features in an experimental state are expected to be less tested and have bugs - it’s essentially a “beta” feature. It has a security bug? Though - you weren’t supposed to be using it in a security-sensitive environment in the first place, it sounds perfectly reasonable to me that it should be addressed in a normal release as opposed to an out-of-band one.
We can argue if forking the project is or isn’t extreme, but the devs absolutely have good reason to be pissed. This is typical management making decisions without understanding technical nuances and - from what is being told by the devs - not talking it through before doing it.
For the record I agree with @fernandofig@reddthat.com, but I also want to add that a DoS is not necessarily a security risk. If it can be leveraged to expose sensitive information, then yes, that’s a vulnerability; this isn’t that.
When NGINX Plus or NGINX OSS are configured to use the HTTP/3 QUIC module, undisclosed requests can cause NGINX worker processes to terminate. (CVE-2024-24989)
Note: The HTTP/3 QUIC module is not enabled by default and is considered experimental. For more information, refer to Support for QUIC and HTTP/3.
#Impact
Traffic is disrupted while the NGINX process restarts. This vulnerability allows a remote unauthenticated attacker to cause a denial-of-service (DoS) on the NGINX system. There is no control plane exposure; this is a data plane issue only.
NGINX Plus R30 P2 and R31 P1
Open source subscription R5 P2 and R6 P1
Open source mainline version 1.25.4
but most importantly:
The latest NGINX Open source stable version 1.24.0 is not affected.
And saving me the hassle of linking and quoting all 5 of the version history pages for the affected products, the uniting factor is: they’re all based on Open Source versions 1.25.*
None of them are using the latest stable version.
It’s not even going to affect most sites, and definitely not ones for whom downtime is a major issue: they would not be using the non-stable version, much less enabling experimental features in a non-stable version.
But the part that irks me the most is the dillution of what a CVE is. Back in the day, it meant “something that can lead to security breaches,” now it just seems to mean “hey guys, I found a bug.” And that’s bad because now you have one of two outcomes: 1. unnecessarily panicking users by leading them to believe their software is a security risk when it isn’t, or 2. compromising the integrity and usability of CVE reports by drowing the important ones in waves of “look guys, the program crashes when I can leverage root privileges to send it SIGKILL!”
If this was just a bug hunter trying to get paid, that’s one thing, but these were internally assigned and disclosed. This was an inside job. And they either ignored or never consulted the actual experts, the ones they have within their own staff: the devs.
Why? To what end? Did they feel left out, what with not having any CVEs since 2022? Does this play some internal political struggle chess move? Do they just hate the idea of clear and unambiguous communication of major security holes to the general public? Are they trying to disrupt their own users’ faith in their paid products? Does someone actually think a DoS is the worst thing that can happen? Is there an upper level manager running their own 1.25 instance that needs this fixed out-of-band?
Note: The HTTP/3 QUIC module is not enabled by default and is considered experimental
Do note that despite not being enabled by default, it is enabled in the official binary packages.
There’s a funny amount of layers to this thing but as far as I’m concerned, if it’s a feature you ship in the default binary packages on your site, that is definitively enough for a CVE even if it’s disabled by default.
Context:
TLDR: The devs don’t like bugs in released software being assigned CVEs, which requires a special security update instead of a standard bugfix included in the regular update cycle.
And nginx’s announcement about these CVEs
I…agree with the “company” I think. This sounds like dev sour grapes but what the company was asking them to do seems better from the customer pov and for cyber security I’m general.
Maybe I’m missing something.
As a developer myself (though not on the level of these guys): sorry, but just, no.
The key point is this:
Emphasis mine. In software, features marked as “experimental” usually are not meant to be used in a production environment, and if they are, it’s in a “do it at your own risk” understanding. Software features in an experimental state are expected to be less tested and have bugs - it’s essentially a “beta” feature. It has a security bug? Though - you weren’t supposed to be using it in a security-sensitive environment in the first place, it sounds perfectly reasonable to me that it should be addressed in a normal release as opposed to an out-of-band one.
We can argue if forking the project is or isn’t extreme, but the devs absolutely have good reason to be pissed. This is typical management making decisions without understanding technical nuances and - from what is being told by the devs - not talking it through before doing it.
For the record I agree with @fernandofig@reddthat.com, but I also want to add that a DoS is not necessarily a security risk. If it can be leveraged to expose sensitive information, then yes, that’s a vulnerability; this isn’t that.
Digging into the CVEs:
CVE-2024-24989:
CVE-2024-24990 basically says the same.
Some choice clauses:
So it doesn’t take down the server nor the parent process, it kills some threads which then… restart.
I was able to find that the affected versions:
NGINX Plus R30 P2 and R31 P1
Open source subscription R5 P2 and R6 P1 Open source mainline version 1.25.4
but most importantly:
And saving me the hassle of linking and quoting all 5 of the version history pages for the affected products, the uniting factor is: they’re all based on Open Source versions 1.25.*
None of them are using the latest stable version.
It’s not even going to affect most sites, and definitely not ones for whom downtime is a major issue: they would not be using the non-stable version, much less enabling experimental features in a non-stable version.
But the part that irks me the most is the dillution of what a CVE is. Back in the day, it meant “something that can lead to security breaches,” now it just seems to mean “hey guys, I found a bug.” And that’s bad because now you have one of two outcomes: 1. unnecessarily panicking users by leading them to believe their software is a security risk when it isn’t, or 2. compromising the integrity and usability of CVE reports by drowing the important ones in waves of “look guys, the program crashes when I can leverage root privileges to send it SIGKILL!”
If this was just a bug hunter trying to get paid, that’s one thing, but these were internally assigned and disclosed. This was an inside job. And they either ignored or never consulted the actual experts, the ones they have within their own staff: the devs.
Why? To what end? Did they feel left out, what with not having any CVEs since 2022? Does this play some internal political struggle chess move? Do they just hate the idea of clear and unambiguous communication of major security holes to the general public? Are they trying to disrupt their own users’ faith in their paid products? Does someone actually think a DoS is the worst thing that can happen? Is there an upper level manager running their own 1.25 instance that needs this fixed out-of-band?
It’s just all so asinine.
Do note that despite not being enabled by default, it is enabled in the official binary packages.
There’s a funny amount of layers to this thing but as far as I’m concerned, if it’s a feature you ship in the default binary packages on your site, that is definitively enough for a CVE even if it’s disabled by default.
Thank you for digging this out. Turns out it’s even worse than what I gleaned from my surface-level take.
Thanks for this. I get now why the devs are pissed.
You’re not missing anything, dude just threw a hissy fit because he’s not the king of his fiefdom anymore.