Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #10072 from gladiac1337/haproxy-2.0.6-openwrt-19.07

Browse source 
[openwrt-19.07] haproxy: Update HAProxy to v2.0.6
This commit is contained in:
Hannu Nyman 2019-09-25 19:11:40 +03:00 committed by GitHub
commit 55c6121536
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
21 changed files with 337 additions and 570 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
net/haproxy/Makefile
View file

@ -10,12 +10,12 @@
include $(TOPDIR)/rules.mk
PKG_NAME:=haproxy
PKG_VERSION:=2.0.5
PKG_VERSION:=2.0.6
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://www.haproxy.org/download/2.0/src
PKG_HASH:=3f2e0d40af66dd6df1dc2f6055d3de106ba62836d77b4c2e497a82a4bdbc5422
PKG_HASH:=01e1da09452010111107ca1b3a8b7f1927731ba0fe4380bacae1c626fdc521e4
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(BUILD_VARIANT)/$(PKG_NAME)-$(PKG_VERSION)
PKG_MAINTAINER:=Thomas Heil <heil@terminal-consulting.de>, \

2
net/haproxy/get-latest-patches.sh
View file

@ -1,7 +1,7 @@
#!/bin/sh
CLONEURL=https://git.haproxy.org/git/haproxy-2.0.git
BASE_TAG=v2.0.5
BASE_TAG=v2.0.6
TMP_REPODIR=tmprepo
PATCHESDIR=patches

34
net/haproxy/patches/000-BUG-MEDIUM-stick-table-Properly-handle-show-table-with-a-data-type-argument.patch Normal file
View file

@ -0,0 +1,34 @@
commit be17bb8fc37db4e12ea24c9480ff6442017ff656
Author: Christopher Faulet <cfaulet@haproxy.com>
Date: Fri Sep 13 15:15:56 2019 +0200
BUG/MEDIUM: stick-table: Properly handle "show table" with a data type argument
Since the commit 1b8e68e8 ("MEDIUM: stick-table: Stop handling stick-tables as
proxies."), the target field into the table context of the CLI applet was not
anymore a pointer to a proxy. It was replaced by a pointer to a stktable. But,
some parts of the code was not updated accordingly. the function
table_prepare_data_request() still tries to cast it to a pointer to a proxy. The
result is totally undefined. With a bit of luck, when the "show table" command
is used with a data type, we failed to find a table and the error "Data type not
stored in this table" is returned. But crashes may also be experienced.
This patch fixes the issue #262. It must be backported to 2.0.
(cherry picked from commit 4e9a83349a9b523830f79f9632ef777ab0bfcc9d)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/stick_table.c b/src/stick_table.c
index 4b04f18c..8528baa0 100644
--- a/src/stick_table.c
+++ b/src/stick_table.c
@@ -3601,8 +3601,7 @@ static int table_prepare_data_request(struct appctx *appctx, char **args)
return 1;
}
- if (!((struct proxy *)appctx->ctx.table.target)->table ||
- !((struct proxy *)appctx->ctx.table.target)->table->data_ofs[appctx->ctx.table.data_type]) {
+ if (!((struct stktable *)appctx->ctx.table.target)->data_ofs[appctx->ctx.table.data_type]) {
appctx->ctx.cli.severity = LOG_ERR;
appctx->ctx.cli.msg = "Data type not stored in this table\n";
appctx->st0 = CLI_ST_PRINT;

46
net/haproxy/patches/000-MINOR-debug-indicate-the-applet-name-when-the-task-is-task_run_applet.patch
View file

@ -1,46 +0,0 @@
commit 3a761682a65e7e7f7baf172f58b15e567a685387
Author: Willy Tarreau <w@1wt.eu>
Date: Wed Aug 21 14:12:19 2019 +0200
MINOR: debug: indicate the applet name when the task is task_run_applet()
This allows to figure what applet is currently being executed (and likely
hung).
(cherry picked from commit a512b02f67a30ab5519d04f8c8b1263415321c85)
[wt: backported to improve troubleshooting when the watchdog fires]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/debug.c b/src/debug.c
index 3077e97c..36cc9e71 100644
--- a/src/debug.c
+++ b/src/debug.c
@@ -90,6 +90,7 @@ void ha_thread_dump(struct buffer *buf, int thr, int calling_tid)
void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
{
const struct stream *s = NULL;
+ const struct appctx __maybe_unused *appctx = NULL;
if (!task) {
chunk_appendf(buf, "0\n");
@@ -110,7 +111,7 @@ void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
task->call_date ? " ns ago" : "");
chunk_appendf(buf, "%s"
- " fct=%p (%s) ctx=%p\n",
+ " fct=%p (%s) ctx=%p",
pfx,
task->process,
task->process == process_stream ? "process_stream" :
@@ -119,6 +120,11 @@ void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
"?",
task->context);
+ if (task->process == task_run_applet && (appctx = task->context))
+ chunk_appendf(buf, "(%s)\n", appctx->applet->name);
+ else
+ chunk_appendf(buf, "\n");
+
if (task->process == process_stream && task->context)
s = (struct stream *)task->context;
else if (task->process == task_run_applet && task->context)

28
net/haproxy/patches/001-BUG-MINOR-mux-h2-Be-sure-to-have-a-connection-to-unsubcribe.patch Normal file
View file

@ -0,0 +1,28 @@
commit 0e01256a314a6f432ab9826dc9b862e8159dbc48
Author: Christopher Faulet <cfaulet@haproxy.com>
Date: Wed Sep 18 11:07:20 2019 +0200
BUG/MINOR: mux-h2: Be sure to have a connection to unsubcribe
When the mux is released, It must own the connection to unsubcribe.
This patch must be backported to 2.0.
(cherry picked from commit 21d849f52fc64c51e5abf5a8bd69f2aeff8b3125)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/mux_h2.c b/src/mux_h2.c
index 984a81bd..e6bfd03d 100644
--- a/src/mux_h2.c
+++ b/src/mux_h2.c
@@ -677,9 +677,9 @@ static void h2_release(struct h2c *h2c)
}
if (h2c->wait_event.tasklet)
tasklet_free(h2c->wait_event.tasklet);
- if (h2c->wait_event.events != 0)
+ if (conn && h2c->wait_event.events != 0)
conn->xprt->unsubscribe(conn, conn->xprt_ctx, h2c->wait_event.events,
- &h2c->wait_event);
+ &h2c->wait_event);
pool_free(pool_head_h2c, h2c);
}

75
net/haproxy/patches/001-MINOR-tools-add-append_prefixed_str.patch
View file

@ -1,75 +0,0 @@
commit fe575b5ca645d6751fba56efa907952eda200b09
Author: Willy Tarreau <w@1wt.eu>
Date: Wed Aug 21 13:17:37 2019 +0200
MINOR: tools: add append_prefixed_str()
This is somewhat related to indent_msg() except that this one places a
known prefix at the beginning of each line, allows to replace the EOL
character, and not to insert a prefix on the first line if not desired.
It works with a normal output buffer/chunk so it doesn't need to allocate
anything nor to modify the input string. It is suitable for use in multi-
line backtraces.
(cherry picked from commit a2c9911ace8537e0a350daf8d981170a001b6c7a)
[wt: backported to improve troubleshooting when the watchdog fires]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/include/common/standard.h b/include/common/standard.h
index 0f4b1870..cdefc9f5 100644
--- a/include/common/standard.h
+++ b/include/common/standard.h
@@ -1238,6 +1238,7 @@ char *memprintf(char **out, const char *format, ...)
* free(err);
*/
char *indent_msg(char **out, int level);
+int append_prefixed_str(struct buffer *out, const char *in, const char *pfx, char eol, int first);
/* removes environment variable <name> from the environment as found in
* environ. This is only provided as an alternative for systems without
diff --git a/src/standard.c b/src/standard.c
index 2f205f74..717c14a9 100644
--- a/src/standard.c
+++ b/src/standard.c
@@ -3709,6 +3709,41 @@ char *indent_msg(char **out, int level)
return ret;
}
+/* makes a copy of message <in> into <out>, with each line prefixed with <pfx>
+ * and end of lines replaced with <eol> if not 0. The first line to indent has
+ * to be indicated in <first> (starts at zero), so that it is possible to skip
+ * indenting the first line if it has to be appended after an existing message.
+ * Empty strings are never indented, and NULL strings are considered empty both
+ * for <in> and <pfx>. It returns non-zero if an EOL was appended as the last
+ * character, non-zero otherwise.
+ */
+int append_prefixed_str(struct buffer *out, const char *in, const char *pfx, char eol, int first)
+{
+ int bol, lf;
+ int pfxlen = pfx ? strlen(pfx) : 0;
+
+ if (!in)
+ return 0;
+
+ bol = 1;
+ lf = 0;
+ while (*in) {
+ if (bol && pfxlen) {
+ if (first > 0)
+ first--;
+ else
+ b_putblk(out, pfx, pfxlen);
+ bol = 0;
+ }
+
+ lf = (*in == '\n');
+ bol |= lf;
+ b_putchr(out, (lf && eol) ? eol : *in);
+ in++;
+ }
+ return lf;
+}
+
/* removes environment variable <name> from the environment as found in
* environ. This is only provided as an alternative for systems without
* unsetenv() (old Solaris and AIX versions). THIS IS NOT THREAD SAFE.

83
net/haproxy/patches/002-BUG-MAJOR-mux-h2-Handle-HEADERS-frames-received-after-a-RST_STREAM-frame.patch Normal file
View file

@ -0,0 +1,83 @@
commit 96b88f2e605e76f2a472cf9fa83398ff242d47bb
Author: Christopher Faulet <cfaulet@haproxy.com>
Date: Mon Sep 23 15:28:20 2019 +0200
BUG/MAJOR: mux-h2: Handle HEADERS frames received after a RST_STREAM frame
As stated in the RFC7540#5.1, an endpoint that receives any frame other than
PRIORITY after receiving a RST_STREAM MUST treat that as a stream error of type
STREAM_CLOSED. However, frames carrying compression state must still be
processed before being dropped to keep the HPACK decoder synchronized. This had
to be the purpose of the commit 8d9ac3ed8b ("BUG/MEDIUM: mux-h2: do not abort
HEADERS frame before decoding them"). But, the test on the frame type was
inverted.
This bug is major because desynchronizing the HPACK decoder leads to mixup
indexed headers in messages. From the time an HEADERS frame is received and
ignored for a closed stream, wrong headers may be sent to the following streams.
This patch may fix several bugs reported on github (#116, #290, #292). It must
be backported to 2.0 and 1.9.
(cherry picked from commit 6884aa3eb00d1a5eb6f9c81a3a00288c13652938)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/mux_h2.c b/src/mux_h2.c
index e6bfd03d..eb773a26 100644
--- a/src/mux_h2.c
+++ b/src/mux_h2.c
@@ -2106,6 +2106,9 @@ static struct h2s *h2c_frt_handle_headers(struct h2c *h2c, struct h2s *h2s)
*/
static struct h2s *h2c_bck_handle_headers(struct h2c *h2c, struct h2s *h2s)
{
+ struct buffer rxbuf = BUF_NULL;
+ unsigned long long body_len = 0;
+ uint32_t flags = 0;
int error;
if (!b_size(&h2c->dbuf))
@@ -2114,7 +2117,18 @@ static struct h2s *h2c_bck_handle_headers(struct h2c *h2c, struct h2s *h2s)
if (b_data(&h2c->dbuf) < h2c->dfl && !b_full(&h2c->dbuf))
return NULL; // incomplete frame
- error = h2c_decode_headers(h2c, &h2s->rxbuf, &h2s->flags, &h2s->body_len);
+ if (h2s->st != H2_SS_CLOSED) {
+ error = h2c_decode_headers(h2c, &h2s->rxbuf, &h2s->flags, &h2s->body_len);
+ }
+ else {
+ /* the connection was already killed by an RST, let's consume
+ * the data and send another RST.
+ */
+ error = h2c_decode_headers(h2c, &rxbuf, &flags, &body_len);
+ h2s_error(h2s, H2_ERR_STREAM_CLOSED);
+ h2c->st0 = H2_CS_FRAME_E;
+ goto send_rst;
+ }
/* unrecoverable error ? */
if (h2c->st0 >= H2_CS_ERROR)
@@ -2150,6 +2164,15 @@ static struct h2s *h2c_bck_handle_headers(struct h2c *h2c, struct h2s *h2s)
}
return h2s;
+
+ send_rst:
+ /* make the demux send an RST for the current stream. We may only
+ * do this if we're certain that the HEADERS frame was properly
+ * decompressed so that the HPACK decoder is still kept up to date.
+ */
+ h2_release_buf(h2c, &rxbuf);
+ h2c->st0 = H2_CS_FRAME_E;
+ return h2s;
}
/* processes a DATA frame. Returns > 0 on success or zero on missing data.
@@ -2459,7 +2482,7 @@ static void h2_process_demux(struct h2c *h2c)
goto strm_err;
}
- if (h2s->flags & H2_SF_RST_RCVD && h2_ft_bit(h2c->dft) & H2_FT_HDR_MASK) {
+ if (h2s->flags & H2_SF_RST_RCVD && !(h2_ft_bit(h2c->dft) & H2_FT_HDR_MASK)) {
/* RFC7540#5.1:closed: an endpoint that
* receives any frame other than PRIORITY after
* receiving a RST_STREAM MUST treat that as a

66
net/haproxy/patches/002-MINOR-lua-export-applet-and-task-handlers.patch
View file

@ -1,66 +0,0 @@
commit 83a5ff403a2cd625832f01032c0feb8bf9c2a89e
Author: Willy Tarreau <w@1wt.eu>
Date: Wed Aug 21 14:14:50 2019 +0200
MINOR: lua: export applet and task handlers
The current functions are seen outside from the debugging code and are
convenient to export so that we can improve the thread dump output :
void hlua_applet_tcp_fct(struct appctx *ctx);
void hlua_applet_http_fct(struct appctx *ctx);
struct task *hlua_process_task(struct task *task, void *context, unsigned short state);
Of course they are only available when USE_LUA is defined.
(cherry picked from commit 60409db0b1743d670e54244425f6e08c389b7dde)
[wt: backported to improve troubleshooting when the watchdog fires;
while in 2.0 we also have hlua_applet_htx_fct(), it's not
visible outside hlua_applet_http_fct() so we don't care]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/include/proto/hlua.h b/include/proto/hlua.h
index 7ad5a99e..32468b77 100644
--- a/include/proto/hlua.h
+++ b/include/proto/hlua.h
@@ -27,6 +27,9 @@
void hlua_ctx_destroy(struct hlua *lua);
void hlua_init();
int hlua_post_init();
+void hlua_applet_tcp_fct(struct appctx *ctx);
+void hlua_applet_http_fct(struct appctx *ctx);
+struct task *hlua_process_task(struct task *task, void *context, unsigned short state);
#else /* USE_LUA */
diff --git a/src/hlua.c b/src/hlua.c
index d2708f87..813aa724 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -6237,7 +6237,7 @@ __LJMP static int hlua_set_nice(lua_State *L)
* Task wrapper are longjmp safe because the only one Lua code
* executed is the safe hlua_ctx_resume();
*/
-static struct task *hlua_process_task(struct task *task, void *context, unsigned short state)
+struct task *hlua_process_task(struct task *task, void *context, unsigned short state)
{
struct hlua *hlua = context;
enum hlua_exec status;
@@ -7045,7 +7045,7 @@ static int hlua_applet_tcp_init(struct appctx *ctx, struct proxy *px, struct str
return 1;
}
-static void hlua_applet_tcp_fct(struct appctx *ctx)
+void hlua_applet_tcp_fct(struct appctx *ctx)
{
struct stream_interface *si = ctx->owner;
struct stream *strm = si_strm(si);
@@ -7417,7 +7417,7 @@ static void hlua_applet_htx_fct(struct appctx *ctx)
goto done;
}
-static void hlua_applet_http_fct(struct appctx *ctx)
+void hlua_applet_http_fct(struct appctx *ctx)
{
struct stream_interface *si = ctx->owner;
struct stream *strm = si_strm(si);

53
net/haproxy/patches/003-BUG-MEDIUM-check-threads-make-external-checks-run-exclusively-on-thread-1.patch Normal file
View file

@ -0,0 +1,53 @@
commit b143711afe833f9824a7372b88ef9435ff240e9a
Author: Willy Tarreau <w@1wt.eu>
Date: Tue Sep 3 18:55:02 2019 +0200
BUG/MEDIUM: check/threads: make external checks run exclusively on thread 1
See GH issues #141 for all the context. In short, registered signal
handlers are not inherited by other threads during startup, which is
normally not a problem, except that we need that the same thread as
the one doing the fork() cleans up the old process using waitpid()
once its death is reported via SIGCHLD, as happens in external checks.
The only simple solution to this at the moment is to make sure that
external checks are exclusively run on the first thread, the one
which registered the signal handlers on startup. It will be far more
than enough anyway given that external checks must not require to be
load balanced on multiple threads! A more complex solution could be
designed over the long term to let each thread deal with all signals
but it sounds overkill.
This must be backported as far as 1.8.
(cherry picked from commit 6dd4ac890b5810b0f0fe81725fda05ad3d052849)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/checks.c b/src/checks.c
index 7b55abda..b088da2e 100644
--- a/src/checks.c
+++ b/src/checks.c
@@ -2175,7 +2175,7 @@ static struct task *process_chk_proc(struct task *t, void *context, unsigned sho
/* a success was detected */
check_notify_success(check);
}
- task_set_affinity(t, MAX_THREADS_MASK);
+ task_set_affinity(t, 1);
check->state &= ~CHK_ST_INPROGRESS;
pid_list_del(check->curpid);
@@ -2423,8 +2423,13 @@ static int start_check_task(struct check *check, int mininter,
int nbcheck, int srvpos)
{
struct task *t;
+ unsigned long thread_mask = MAX_THREADS_MASK;
+
+ if (check->type == PR_O2_EXT_CHK)
+ thread_mask = 1;
+
/* task for the check */
- if ((t = task_new(MAX_THREADS_MASK)) == NULL) {
+ if ((t = task_new(thread_mask)) == NULL) {
ha_alert("Starting [%s:%s] check: out of memory.\n",
check->server->proxy->id, check->server->id);
return 0;

85
net/haproxy/patches/003-MEDIUM-debug-make-the-thread-dump-code-show-Lua-backtraces.patch
View file

@ -1,85 +0,0 @@
commit 4856b36cba80a259a78645753520323caca78d0f
Author: Willy Tarreau <w@1wt.eu>
Date: Wed Aug 21 14:16:02 2019 +0200
MEDIUM: debug: make the thread dump code show Lua backtraces
When we dump a thread's state (show thread, panic) we don't know if
anything is happening in Lua, which can be problematic especially when
calling external functions. With this patch, the thread dump code can
now detect if we're running in a global Lua task (hlua_process_task),
or in a TCP or HTTP Lua service (task_run_applet and applet.fct ==
hlua_applet_tcp_fct or http_applet_http_fct), or a fetch/converter
from an analyser (s->hlua != NULL). In such situations, it's able to
append a formatted Lua backtrace of the Lua execution path with
function names, file names and line numbers.
Note that a shorter alternative could be to call "luaL_where(hlua->T,0)"
which only prints the current location, but it's not necessarily sufficient
for complex code.
(cherry picked from commit 78a7cb648ca33823c06430cedc6859ea7e7cd5df)
[wt: backported to improve troubleshooting when the watchdog fires]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/debug.c b/src/debug.c
index 36cc9e71..79bea884 100644
--- a/src/debug.c
+++ b/src/debug.c
@@ -26,6 +26,7 @@
#include <proto/cli.h>
#include <proto/fd.h>
+#include <proto/hlua.h>
#include <proto/stream_interface.h>
#include <proto/task.h>
@@ -91,6 +92,7 @@ void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
{
const struct stream *s = NULL;
const struct appctx __maybe_unused *appctx = NULL;
+ struct hlua __maybe_unused *hlua = NULL;
if (!task) {
chunk_appendf(buf, "0\n");
@@ -117,6 +119,9 @@ void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
task->process == process_stream ? "process_stream" :
task->process == task_run_applet ? "task_run_applet" :
task->process == si_cs_io_cb ? "si_cs_io_cb" :
+#ifdef USE_LUA
+ task->process == hlua_process_task ? "hlua_process_task" :
+#endif
"?",
task->context);
@@ -134,6 +139,30 @@ void ha_task_dump(struct buffer *buf, const struct task *task, const char *pfx)
if (s)
stream_dump(buf, s, pfx, '\n');
+
+#ifdef USE_LUA
+ hlua = NULL;
+ if (s && (hlua = s->hlua)) {
+ chunk_appendf(buf, "%sCurrent executing Lua from a stream analyser -- ", pfx);
+ }
+ else if (task->process == hlua_process_task && (hlua = task->context)) {
+ chunk_appendf(buf, "%sCurrent executing a Lua task -- ", pfx);
+ }
+ else if (task->process == task_run_applet && (appctx = task->context) &&
+ (appctx->applet->fct == hlua_applet_tcp_fct && (hlua = appctx->ctx.hlua_apptcp.hlua))) {
+ chunk_appendf(buf, "%sCurrent executing a Lua TCP service -- ", pfx);
+ }
+ else if (task->process == task_run_applet && (appctx = task->context) &&
+ (appctx->applet->fct == hlua_applet_http_fct && (hlua = appctx->ctx.hlua_apphttp.hlua))) {
+ chunk_appendf(buf, "%sCurrent executing a Lua HTTP service -- ", pfx);
+ }
+
+ if (hlua) {
+ luaL_traceback(hlua->T, hlua->T, NULL, 0);
+ if (!append_prefixed_str(buf, lua_tostring(hlua->T, -1), pfx, '\n', 1))
+ b_putchr(buf, '\n');
+ }
+#endif
}
/* This function dumps all profiling settings. It returns 0 if the output

79
net/haproxy/patches/004-BUG-MEDIUM-mux-h1-do-not-truncate-trailing-0CRLF-on-buffer-boundary.patch
View file

@ -1,79 +0,0 @@
commit 9a408abbb8559df5718bc696bd9c3934c6500d63
Author: Willy Tarreau <w@1wt.eu>
Date: Fri Aug 23 08:11:36 2019 +0200
BUG/MEDIUM: mux-h1: do not truncate trailing 0CRLF on buffer boundary
The H1 message parser calls the various message block parsers with an
offset indicating where in the buffer to start from, and only consumes
the data at the end of the parsing. The headers and trailers parsers
have a condition detecting if a headers or trailers block is too large
to fit into the buffer. This is detected by an incomplete block while
the buffer is full. Unfortunately it doesn't take into account the fact
that the block may be parsed after other blocks that are still present
in the buffer, resulting in aborting some transfers early as reported
in issue #231. This typically happens if a trailers block is incomplete
at the end of a buffer full of data, which typically happens with data
sizes multiple of the buffer size minus less than the trailers block
size. It also happens with the CRLF that follows the 0-sized chunk of
any transfer-encoded contents is itself on such a boundary since this
CRLF is technically part of the trailers block. This can be reproduced
by asking a server to retrieve exactly 31532 or 31533 bytes of static
data using chunked encoding with curl, which reports:
transfer closed with outstanding read data remaining
This issue was revealed in 2.0 and does not affect 1.9 because in 1.9
the trailers block was processed at once as part of the data block
processing, and would simply give up and wait for the rest of the data
to arrive.
It's interesting to note that the headers block parsing is also affected
by this issue but in practice it has a much more limited impact since a
headers block is normally only parsed at the beginning of a buffer. The
only case where it seems to matter is when dealing with a response buffer
full of 100-continue header blocks followed by a regular header block,
which will then be rejected for the same reason.
This fix must be backported to 2.0 and partially to 1.9 (the headers
block part).
(cherry picked from commit 347f464d4e5a8a2bf3acd2411a6c8228e605e7f6)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/mux_h1.c b/src/mux_h1.c
index fa694c41..01f225a2 100644
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -995,10 +995,11 @@ static size_t h1_process_headers(struct h1s *h1s, struct h1m *h1m, struct htx *h
ret = h1_headers_to_hdr_list(b_peek(buf, *ofs), b_tail(buf),
hdrs, sizeof(hdrs)/sizeof(hdrs[0]), h1m, &h1sl);
if (ret <= 0) {
- /* Incomplete or invalid message. If the buffer is full, it's an
- * error because headers are too large to be handled by the
- * parser. */
- if (ret < 0 || (!ret && !buf_room_for_htx_data(buf)))
+ /* Incomplete or invalid message. If the input buffer only
+ * contains headers and is full, which is detected by it being
+ * full and the offset to be zero, it's an error because
+ * headers are too large to be handled by the parser. */
+ if (ret < 0 || (!ret && !*ofs && !buf_room_for_htx_data(buf)))
goto error;
goto end;
}
@@ -1339,10 +1340,11 @@ static size_t h1_process_trailers(struct h1s *h1s, struct h1m *h1m, struct htx *
ret = h1_headers_to_hdr_list(b_peek(buf, *ofs), b_tail(buf),
hdrs, sizeof(hdrs)/sizeof(hdrs[0]), &tlr_h1m, NULL);
if (ret <= 0) {
- /* Incomplete or invalid trailers. If the buffer is full, it's
- * an error because traliers are too large to be handled by the
- * parser. */
- if (ret < 0 || (!ret && !buf_room_for_htx_data(buf)))
+ /* Incomplete or invalid trailers. If the input buffer only
+ * contains trailers and is full, which is detected by it being
+ * full and the offset to be zero, it's an error because
+ * trailers are too large to be handled by the parser. */
+ if (ret < 0 || (!ret && !*ofs && !buf_room_for_htx_data(buf)))
goto error;
goto end;
}

43
net/haproxy/patches/004-BUG-MINOR-stream-int-Process-connection-CS-errors-first-in-si_cs_send.patch Normal file
View file

@ -0,0 +1,43 @@
commit cfdef9f428869f1570d51a5bd8975d8a42f31eab
Author: Christopher Faulet <cfaulet@haproxy.com>
Date: Mon Sep 23 15:57:29 2019 +0200
BUG/MINOR: stream-int: Process connection/CS errors first in si_cs_send()
Errors on the connections or the conn-stream must always be processed in
si_cs_send(), even if the stream-interface is already subscribed on
sending. This patch does not fix any concrete bug per-se. But it is required by
the following one to handle those errors during synchronous sends.
This patch must be backported with the following one to 2.0 and probably to 1.9
too, but with caution because the code is really different.
(cherry picked from commit 328ed220a8c5211aa8b6f37b982f319cf6b3f3d1)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/stream_interface.c b/src/stream_interface.c
index 7d89cc90..4130444e 100644
--- a/src/stream_interface.c
+++ b/src/stream_interface.c
@@ -652,10 +652,6 @@ int si_cs_send(struct conn_stream *cs)
int ret;
int did_send = 0;
- /* We're already waiting to be able to send, give up */
- if (si->wait_event.events & SUB_RETRY_SEND)
- return 0;
-
if (conn->flags & CO_FL_ERROR || cs->flags & (CS_FL_ERROR|CS_FL_ERR_PENDING)) {
/* We're probably there because the tasklet was woken up,
* but process_stream() ran before, detected there were an
@@ -669,6 +665,10 @@ int si_cs_send(struct conn_stream *cs)
return 1;
}
+ /* We're already waiting to be able to send, give up */
+ if (si->wait_event.events & SUB_RETRY_SEND)
+ return 0;
+
/* we might have been called just after an asynchronous shutw */
if (conn->flags & CO_FL_SOCK_WR_SH || oc->flags & CF_SHUTW)
return 1;

68
net/haproxy/patches/005-BUG-MEDIUM-mux-h1-do-not-report-errors-on-transfers-ending-on-buffer-full.patch
View file

@ -1,68 +0,0 @@
commit 620381599324e15403002270637a3b677c3fe7e5
Author: Willy Tarreau <w@1wt.eu>
Date: Fri Aug 23 09:29:29 2019 +0200
BUG/MEDIUM: mux-h1: do not report errors on transfers ending on buffer full
If a receipt ends with the HTX buffer full and everything is completed except
appending the HTX EOM block, we end up detecting an error because the H1
parser did not switch to H1_MSG_DONE yet while all conditions for an end of
stream and end of buffer are met. This can be detected by retrieving 31532
or 31533 chunk-encoded bytes over H1 and seeing haproxy log "SD--" at the
end of a successful transfer.
Ideally the EOM part should be totally independent on the H1 message state
since the block was really parsed and finished. So we should switch to a
last state requiring to send only EOM. However this needs a few risky
changes. This patch aims for simplicity and backport safety, thus it only
adds a flag to the H1 stream indicating that an EOM is still needed, and
excludes this condition from the ones used to detect end of processing. A
cleaner approach needs to be studied, either by adding a state before DONE
or by setting DONE once the various blocks are parsed and before trying to
send EOM.
This fix must be backported to 2.0. The issue does not seem to affect 1.9
though it is not yet known why, probably that it is related to the different
encoding of trailers which always leaves a bit of room to let EOM be stored.
(cherry picked from commit 0bb5a5c4b5ad375b1254c2e8bec2dd5ea85d6ebb)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/mux_h1.c b/src/mux_h1.c
index 01f225a2..b9a37ce5 100644
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -67,7 +67,8 @@
#define H1S_F_BUF_FLUSH 0x00000100 /* Flush input buffer and don't read more data */
#define H1S_F_SPLICED_DATA 0x00000200 /* Set when the kernel splicing is in used */
#define H1S_F_HAVE_I_TLR 0x00000800 /* Set during input process to know the trailers were processed */
-/* 0x00001000 .. 0x00002000 unused */
+#define H1S_F_APPEND_EOM 0x00001000 /* Send EOM to the HTX buffer */
+/* 0x00002000 .. 0x00002000 unused */
#define H1S_F_HAVE_O_CONN 0x00004000 /* Set during output process to know connection mode was processed */
/* H1 connection descriptor */
@@ -954,9 +955,12 @@ static size_t h1_eval_htx_res_size(struct h1m *h1m, union h1_sl *h1sl, struct ht
*/
static size_t h1_process_eom(struct h1s *h1s, struct h1m *h1m, struct htx *htx, size_t max)
{
- if (max < sizeof(struct htx_blk) + 1 || !htx_add_endof(htx, HTX_BLK_EOM))
+ if (max < sizeof(struct htx_blk) + 1 || !htx_add_endof(htx, HTX_BLK_EOM)) {
+ h1s->flags |= H1S_F_APPEND_EOM;
return 0;
+ }
+ h1s->flags &= ~H1S_F_APPEND_EOM;
h1m->state = H1_MSG_DONE;
h1s->cs->flags |= CS_FL_EOI;
return (sizeof(struct htx_blk) + 1);
@@ -1472,7 +1476,8 @@ static size_t h1_process_input(struct h1c *h1c, struct buffer *buf, size_t count
else if (h1s_data_pending(h1s) && !htx_is_empty(htx))
h1s->cs->flags |= CS_FL_RCV_MORE | CS_FL_WANT_ROOM;
- if ((h1s->flags & H1S_F_REOS) && (!h1s_data_pending(h1s) || htx_is_empty(htx))) {
+ if (((h1s->flags & (H1S_F_REOS|H1S_F_APPEND_EOM)) == H1S_F_REOS) &&
+ (!h1s_data_pending(h1s) || htx_is_empty(htx))) {
h1s->cs->flags |= CS_FL_EOS;
if (h1m->state > H1_MSG_LAST_LF && h1m->state < H1_MSG_DONE)
h1s->cs->flags |= CS_FL_ERROR;

40
net/haproxy/patches/005-BUG-MEDIUM-stream-int-Process-connection-CS-errors-during-synchronous-sends.patch Normal file
View file

@ -0,0 +1,40 @@
commit a80c11777b09ea494b5da76a5bcb096851fb6097
Author: Christopher Faulet <cfaulet@haproxy.com>
Date: Mon Sep 23 16:11:57 2019 +0200
BUG/MEDIUM: stream-int: Process connection/CS errors during synchronous sends
If an error occurred on the connection or the conn-stream, no syncrhonous send
is performed. If the error was not already processed and there is no more I/O,
it will never be processed and the stream will never be notified of this
error. This may block the stream until a timeout is reached or infinitly if
there is no timeout.
Concretly, this bug can be triggered time to time with h2spec, running the test
"http2/5.1.1/2".
This patch depends on the commit 328ed220a "BUG/MINOR: stream-int: Process
connection/CS errors first in si_cs_send()". Both must be backported to 2.0 and
probably to 1.9. In 1.9, the code is totally different, so this patch would have
to be adapted.
(cherry picked from commit e55a5a41713b629d349ba020183744a38129b892)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/src/stream_interface.c b/src/stream_interface.c
index 4130444e..ef0fea7f 100644
--- a/src/stream_interface.c
+++ b/src/stream_interface.c
@@ -922,12 +922,6 @@ void si_sync_send(struct stream_interface *si)
if (!cs)
return;
- if (cs->flags & (CS_FL_ERROR|CS_FL_ERR_PENDING))
- return;
-
- if (cs->conn->flags & CO_FL_ERROR)
- return;
-
si_cs_send(cs);
}

53
net/haproxy/patches/006-BUG-MEDIUM-checks-make-sure-the-connection-is-ready-before-trying-to-recv.patch Normal file
View file

@ -0,0 +1,53 @@
commit 489bbd321e46c110ab9d92fb91725870d7c40491
Author: Willy Tarreau <w@1wt.eu>
Date: Tue Sep 24 10:43:03 2019 +0200
BUG/MEDIUM: checks: make sure the connection is ready before trying to recv
As identified in issue #278, the backport of commit c594039225 ("BUG/MINOR:
checks: do not uselessly poll for reads before the connection is up")
introduced a regression in 2.0 when default checks are enabled (not
"option tcp-check"), but it did not affect 2.1.
What happens is that in 2.0 and earlier we have the fd cache which makes
a speculative call to the I/O functions after an attempt to connect, and
the __event_srv_chk_r() function was absolutely not designed to be called
while a connection attempt is still pending. Thus what happens is that the
test for success/failure expects the verdict to be final before waking up
the check task, and since the connection is not yet validated, it fails.
It will usually work over the loopback depending on scheduling, which is
why it doesn't fail in reg tests.
In 2.1 after the failed connect(), we subscribe to polling and usually come
back with a validated connection, so the function is not expected to be
called before it completes, except if it happens as a side effect of some
spurious wake calls, which should not have any effect on such a check.
The other check types are not impacted by this issue because they all
check for a minimum data length in the buffer, and wait for more data
until they are satisfied.
This patch fixes the issue by explicitly checking that the connection
is established before trying to read or to give a verdict. This way the
function becomes safe to call regardless of the connection status (even
if it's still totally ugly).
This fix must be backported to 2.0.
(cherry picked from commit 0f0393fc0d2badc5ea329844691f06ba28827f78)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/checks.c b/src/checks.c
index b088da2e..06f47ad9 100644
--- a/src/checks.c
+++ b/src/checks.c
@@ -875,6 +875,9 @@ static void __event_srv_chk_r(struct conn_stream *cs)
}
}
+ /* the rest of the code below expects the connection to be ready! */
+ if (!(conn->flags & CO_FL_CONNECTED) && !done)
+ goto wait_more_data;
/* Intermediate or complete response received.
* Terminate string in b_head(&check->bi) buffer.

27
net/haproxy/patches/006-DOC-fixed-typo-in-management-txt.patch
View file

@ -1,27 +0,0 @@
commit 7c80af0fb53f2a1d93a597f7d97cc67996e36be2
Author: n9@users.noreply.github.com <n9@users.noreply.github.com>
Date: Fri Aug 23 11:21:05 2019 +0200
DOC: fixed typo in management.txt
replaced fot -> for
added two periods
(cherry picked from commit 25a1c8e4539c12c19a3fe04aabe563cdac5e36db)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/doc/management.txt b/doc/management.txt
index 616a040b..ad6011e5 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1549,8 +1549,8 @@ enable agent <backend>/<server>
level "admin".
enable dynamic-cookie backend <backend>
- Enable the generation of dynamic cookies fot the backend <backend>
- A secret key must also be provided
+ Enable the generation of dynamic cookies for the backend <backend>.
+ A secret key must also be provided.
enable frontend <frontend>
Resume a frontend which was temporarily stopped. It is possible that some of

35
net/haproxy/patches/007-BUG-MINOR-mworker-disable-SIGPROF-on-re-exec.patch
View file

@ -1,35 +0,0 @@
commit f259fcc00a04e633a7a64f894a719f78f3644867
Author: Willy Tarreau <w@1wt.eu>
Date: Mon Aug 26 10:37:39 2019 +0200
BUG/MINOR: mworker: disable SIGPROF on re-exec
If haproxy is built with profiling enabled with -pg, it is possible to
see the master quit during a reload while it's re-executing itself with
error code 155 (signal 27) saying "Profile timer expired)". This happens
if the SIGPROF signal is delivered during the execve() call while the
handler was already unregistered. The issue itself is not directly inside
haproxy but it's easy to address. This patch disables this signal before
calling execvp() during a master reload. A simple test for this consists
in running this little script with haproxy started in master-worker mode :
$ while usleep 50000; do killall -USR2 haproxy; done
This fix should be backported to all versions using the master-worker
model.
(cherry picked from commit e0d86e2c1caaaa2141118e3309d479de5f67e855)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/haproxy.c b/src/haproxy.c
index f6f00fc1..c93b0d13 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -695,6 +695,7 @@ void mworker_reload()
}
ha_warning("Reexecuting Master process\n");
+ signal(SIGPROF, SIG_IGN);
execvp(next_argv[0], next_argv);
ha_warning("Failed to reexecute the master process [%d]: %s\n", pid, strerror(errno));

0
net/haproxy/patches/010-OPENWRT-add-uclibc-support.patch → net/haproxy/patches/007-OPENWRT-add-uclibc-support.patch
View file

52
net/haproxy/patches/008-BUG-MEDIUM-listener-threads-fix-an-AB-BA-locking-issue-in-delete_listener.patch
View file

@ -1,52 +0,0 @@
commit b10c8d7641cc8ceae6fba4506b7f987d66109bd9
Author: Willy Tarreau <w@1wt.eu>
Date: Mon Aug 26 10:55:52 2019 +0200
BUG/MEDIUM: listener/threads: fix an AB/BA locking issue in delete_listener()
The delete_listener() function takes the listener's lock before taking
the proto_lock, which is contrary to what other functions do, possibly
causing an AB/BA deadlock. In practice the two only places where both
are taken are during protocol_enable_all() and delete_listener(), the
former being used during startup and the latter during stop. In practice
during reload floods, it is technically possible for a thread to be
initializing the listeners while another one is stopping. While this
is too hard to trigger on 2.0 and above due to the synchronization of
all threads during startup, it's reasonably easy to do in 1.9 by having
hundreds of listeners, starting 64 threads and flooding them with reloads
like this :
$ while usleep 50000; do killall -USR2 haproxy; done
Usually in less than a minute, all threads will be deadlocked. The fix
consists in always taking the proto_lock before the listener lock. It
seems to be the only place where these two locks were reversed. This
fix needs to be backported to 2.0, 1.9, and 1.8.
(cherry picked from commit 6ee9f8df3bfbb811526cff3313da5758b1277bc6)
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/listener.c b/src/listener.c
index b5fe2ac2..54c09960 100644
--- a/src/listener.c
+++ b/src/listener.c
@@ -595,17 +595,17 @@ int create_listeners(struct bind_conf *bc, const struct sockaddr_storage *ss,
*/
void delete_listener(struct listener *listener)
{
+ HA_SPIN_LOCK(PROTO_LOCK, &proto_lock);
HA_SPIN_LOCK(LISTENER_LOCK, &listener->lock);
if (listener->state == LI_ASSIGNED) {
listener->state = LI_INIT;
- HA_SPIN_LOCK(PROTO_LOCK, &proto_lock);
LIST_DEL(&listener->proto_list);
listener->proto->nb_listeners--;
- HA_SPIN_UNLOCK(PROTO_LOCK, &proto_lock);
_HA_ATOMIC_SUB(&jobs, 1);
_HA_ATOMIC_SUB(&listeners, 1);
}
HA_SPIN_UNLOCK(LISTENER_LOCK, &listener->lock);
+ HA_SPIN_UNLOCK(PROTO_LOCK, &proto_lock);
}
/* Returns a suitable value for a listener's backlog. It uses the listener's,

0
net/haproxy/patches/011-OPENWRT-openssl-deprecated.patch → net/haproxy/patches/008-OPENWRT-openssl-deprecated.patch
View file

34
net/haproxy/patches/009-BUG-MEDIUM-url32-does-not-take-the-path-part-into-account-in-the-returned-hash.patch
View file

@ -1,34 +0,0 @@
commit 4db294bc0b7988607f2dfdb9d57974b2ba47cbc3
Author: Jerome Magnin <jmagnin@haproxy.com>
Date: Mon Aug 26 11:44:21 2019 +0200
BUG/MEDIUM: url32 does not take the path part into account in the returned hash.
The url32 sample fetch does not take the path part of the URL into
account. This is because in smp_fetch_url32() we erroneously modify
path.len and path.ptr before testing their value and building the
path based part of the hash.
This fixes issue #235
This must be backported as far as 1.9, when HTX was introduced.
(cherry picked from commit 2dd26ca9ff8e642611b8b012d6aee45ea45196bc)
[wt: adjusted context, we still have legacy in 2.0]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/src/http_fetch.c b/src/http_fetch.c
index e372a122..6448bde9 100644
--- a/src/http_fetch.c
+++ b/src/http_fetch.c
@@ -2735,10 +2735,6 @@ static int smp_fetch_url32(const struct arg *args, struct sample *smp, const cha
/* now retrieve the path */
sl = http_get_stline(htx);
path = http_get_path(htx_sl_req_uri(sl));
- while (path.len > 0 && *(path.ptr) != '?') {
- path.ptr++;
- path.len--;
- }
if (path.len && *(path.ptr) == '/') {
while (path.len--)
hash = *(path.ptr++) + (hash << 6) + (hash << 16) - hash;