Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

haproxy: fixes from upstream

Browse source 
- [PATCH 22/25] DOC: minor fix on {sc,src}_kbytes_{in,out}
 - [PATCH 23/25] DOC: fix alphabetical sort of converters
 - [PATCH 24/25] BUG/MAJOR: http: correctly rewind the request body
 - [PATCH 25/25] DOC: remove references to CPU=native in the README

Signed-off-by: Thomas Heil <heil@terminal-consulting.de>
This commit is contained in:
Thomas Heil 2014-07-12 13:45:36 +02:00
parent 08379648ee
commit 10d9b68c49
5 changed files with 375 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
net/haproxy/Makefile
View file

@ -10,7 +10,7 @@ include $(TOPDIR)/rules.mk
PKG_NAME:=haproxy
PKG_VERSION:=1.5.1
PKG_RELEASE:=21
PKG_RELEASE:=25
PKG_SOURCE:=haproxy-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=http://haproxy.1wt.eu/download/1.5/src/
PKG_MD5SUM:=49640cf3ddd793a05fbd3394481a1ed4

75
net/haproxy/patches/0022-DOC-minor-fix-on-sc-src-_kbytes_-in-out.patch Normal file
View file

@ -0,0 +1,75 @@
From 911780c5f0e20760164cb0f9b92318185651237c Mon Sep 17 00:00:00 2001
From: Willy Tarreau <w@1wt.eu>
Date: Thu, 10 Jul 2014 15:29:24 +0200
Subject: [PATCH 22/25] DOC: minor fix on {sc,src}_kbytes_{in,out}
These ones report total amount of bytes, not byte rates.
This fix should be backported into 1.5 which has the same error.
(cherry picked from commit a01b974d5f5a067d99f288dcb3e05b78fe780a76)
---
doc/configuration.txt | 35 ++++++++++++++++-------------------
1 file changed, 16 insertions(+), 19 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 8407500..b6d1b3b 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -10386,19 +10386,17 @@ sc_kbytes_in(<ctr>[,<table>]) : integer
sc0_kbytes_in([<table>]) : integer
sc1_kbytes_in([<table>]) : integer
sc2_kbytes_in([<table>]) : integer
- Returns the amount of client-to-server data from the currently tracked
- counters, measured in kilobytes over the period configured in the table. The
- test is currently performed on 32-bit integers, which limits values to 4
- terabytes. See also src_kbytes_in.
+ Returns the total amount of client-to-server data from the currently tracked
+ counters, measured in kilobytes. The test is currently performed on 32-bit
+ integers, which limits values to 4 terabytes. See also src_kbytes_in.
sc_kbytes_out(<ctr>[,<table>]) : integer
sc0_kbytes_out([<table>]) : integer
sc1_kbytes_out([<table>]) : integer
sc2_kbytes_out([<table>]) : integer
- Returns the amount of server-to-client data from the currently tracked
- counters, measured in kilobytes over the period configured in the table. The
- test is currently performed on 32-bit integers, which limits values to 4
- terabytes. See also src_kbytes_out.
+ Returns the total amount of server-to-client data from the currently tracked
+ counters, measured in kilobytes. The test is currently performed on 32-bit
+ integers, which limits values to 4 terabytes. See also src_kbytes_out.
sc_sess_cnt(<ctr>[,<table>]) : integer
sc0_sess_cnt([<table>]) : integer
@@ -10562,19 +10560,18 @@ src_inc_gpc0([<table>]) : integer
tcp-request connection reject if abuse kill
src_kbytes_in([<table>]) : integer
- Returns the amount of data received from the incoming connection's source
- address in the current proxy's stick-table or in the designated stick-table,
- measured in kilobytes over the period configured in the table. If the address
- is not found, zero is returned. The test is currently performed on 32-bit
- integers, which limits values to 4 terabytes. See also
- sc/sc0/sc1/sc2_kbytes_in.
+ Returns the total amount of data received from the incoming connection's
+ source address in the current proxy's stick-table or in the designated
+ stick-table, measured in kilobytes. If the address is not found, zero is
+ returned. The test is currently performed on 32-bit integers, which limits
+ values to 4 terabytes. See also sc/sc0/sc1/sc2_kbytes_in.
src_kbytes_out([<table>]) : integer
- Returns the amount of data sent to the incoming connection's source address
- in the current proxy's stick-table or in the designated stick-table, measured
- in kilobytes over the period configured in the table. If the address is not
- found, zero is returned. The test is currently performed on 32-bit integers,
- which limits values to 4 terabytes. See also sc/sc0/sc1/sc2_kbytes_out.
+ Returns the total amount of data sent to the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table,
+ measured in kilobytes. If the address is not found, zero is returned. The
+ test is currently performed on 32-bit integers, which limits values to 4
+ terabytes. See also sc/sc0/sc1/sc2_kbytes_out.
src_port : integer
Returns an integer value corresponding to the TCP source port of the
--
1.8.5.5

84
net/haproxy/patches/0023-DOC-fix-alphabetical-sort-of-converters.patch Normal file
View file

@ -0,0 +1,84 @@
From 644d9ef5af4f8010412007374c345f7465c97391 Mon Sep 17 00:00:00 2001
From: Willy Tarreau <w@1wt.eu>
Date: Thu, 10 Jul 2014 16:29:08 +0200
Subject: [PATCH 23/25] DOC: fix alphabetical sort of converters
For an unknown reason, these ones were not sorted.
(cherry picked from commit ffcb2e4b42acd710121a57eb39651a373d904e5b)
---
doc/configuration.txt | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index b6d1b3b..f8199b9 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -9887,28 +9887,12 @@ base64
transfer binary content in a way that can be reliably transferred (eg:
an SSL ID can be copied in a header).
-lower
- Convert a string sample to lower case. This can only be placed after a string
- sample fetch function or after a transformation keyword returning a string
- type. The result is of type string.
-
-upper
- Convert a string sample to upper case. This can only be placed after a string
- sample fetch function or after a transformation keyword returning a string
- type. The result is of type string.
-
hex
Converts a binary input sample to an hex string containing two hex digits per
input byte. It is used to log or transfer hex dumps of some binary input data
in a way that can be reliably transferred (eg: an SSL ID can be copied in a
header).
-ipmask(<mask>)
- Apply a mask to an IPv4 address, and use the result for lookups and storage.
- This can be used to make all hosts within a certain mask to share the same
- table entries and as such use the same server. The mask can be passed in
- dotted form (eg: 255.255.255.0) or in CIDR form (eg: 24).
-
http_date([<offset>])
Converts an integer supposed to contain a date since epoch to a string
representing this date in a format suitable for use in HTTP header fields. If
@@ -9917,6 +9901,12 @@ http_date([<offset>])
emit Date header fields, Expires values in responses when combined with a
positive offset, or Last-Modified values when the offset is negative.
+ipmask(<mask>)
+ Apply a mask to an IPv4 address, and use the result for lookups and storage.
+ This can be used to make all hosts within a certain mask to share the same
+ table entries and as such use the same server. The mask can be passed in
+ dotted form (eg: 255.255.255.0) or in CIDR form (eg: 24).
+
language(<value>[,<default>])
Returns the value with the highest q-factor from a list as extracted from the
"accept-language" header using "req.fhdr". Values with no q-factor have a
@@ -9944,6 +9934,11 @@ language(<value>[,<default>])
use_backend english if en
default_backend choose_your_language
+lower
+ Convert a string sample to lower case. This can only be placed after a string
+ sample fetch function or after a transformation keyword returning a string
+ type. The result is of type string.
+
map(<map_file>[,<default_value>])
map_<match_type>(<map_file>[,<default_value>])
map_<match_type>_<output_type>(<map_file>[,<default_value>])
@@ -10001,6 +9996,11 @@ map_<match_type>_<output_type>(<map_file>[,<default_value>])
| `---------------------------- key
`------------------------------------ leading spaces ignored
+upper
+ Convert a string sample to upper case. This can only be placed after a string
+ sample fetch function or after a transformation keyword returning a string
+ type. The result is of type string.
+
7.3.2. Fetching samples from internal states
--------------------------------------------
--
1.8.5.5

167
net/haproxy/patches/0024-BUG-MAJOR-http-correctly-rewind-the-request-body-aft.patch Normal file
View file

@ -0,0 +1,167 @@
From 5bebcd06287be9024f0fba25f350393f02e050c1 Mon Sep 17 00:00:00 2001
From: Willy Tarreau <w@1wt.eu>
Date: Thu, 10 Jul 2014 19:06:10 +0200
Subject: [PATCH 24/25] BUG/MAJOR: http: correctly rewind the request body
after start of forwarding
Daniel Dubovik reported an interesting bug showing that the request body
processing was still not 100% fixed. If a POST request contained short
enough data to be forwarded at once before trying to establish the
connection to the server, we had no way to correctly rewind the body.
The first visible case is that balancing on a header does not always work
on such POST requests since the header cannot be found. But there are even
nastier implications which are that http-send-name-header would apply to
the wrong location and possibly even affect part of the request's body
due to an incorrect rewinding.
There are two options to fix the problem :
- first one is to force the HTTP_MSG_F_WAIT_CONN flag on all hash-based
balancing algorithms and http-send-name-header, but there's always a
risk that any new algorithm forgets to set it ;
- the second option is to account for the amount of skipped data before
the connection establishes so that we always know the position of the
request's body relative to the buffer's origin.
The second option is much more reliable and fits very well in the spirit
of the past changes to fix forwarding. Indeed, at the moment we have
msg->sov which points to the start of the body before headers are forwarded
and which equals zero afterwards (so it still points to the start of the
body before forwarding data). A minor change consists in always making it
point to the start of the body even after data have been forwarded. It means
that it can get a negative value (so we need to change its type to signed)..
In order to avoid wrapping, we only do this as long as the other side of
the buffer is not connected yet.
Doing this definitely fixes the issues above for the requests. Since the
response cannot be rewound we don't need to perform any change there.
This bug was introduced/remained unfixed in 1.5-dev23 so the fix must be
backported to 1.5.
(cherry picked from commit bb2e669f9e73531ac9cc9277b40066b701eec918)
---
doc/internals/body-parsing.txt | 20 +++++++++++++-------
include/types/proto_http.h | 11 ++++++-----
src/proto_http.c | 9 +++++++--
3 files changed, 26 insertions(+), 14 deletions(-)
diff --git a/doc/internals/body-parsing.txt b/doc/internals/body-parsing.txt
index e9c8b4b..5baa549 100644
--- a/doc/internals/body-parsing.txt
+++ b/doc/internals/body-parsing.txt
@@ -67,12 +67,17 @@ msg.next : points to the next byte to inspect. This offset is automatically
automatically adjusted to the number of bytes already inspected.
msg.sov : start of value. First character of the header's value in the header
- states, start of the body in the data states until headers are
- forwarded. This offset is automatically adjusted when inserting or
- removing some headers. In data states, it always constains the size
- of the whole HTTP headers (including the trailing CRLF) that needs
- to be forwarded before the first byte of body. Once the headers are
- forwarded, this value drops to zero.
+ states, start of the body in the data states. Strictly positive
+ values indicate that headers were not forwarded yet (<buf.p> is
+ before the start of the body), and null or positive values are seen
+ after headers are forwarded (<buf.p> is at or past the start of the
+ body). The value stops changing when data start to leave the buffer
+ (in order to avoid integer overflows). So the maximum possible range
+ is -<buf.size> to +<buf.size>. This offset is automatically adjusted
+ when inserting or removing some headers. It is useful to rewind the
+ request buffer to the beginning of the body at any phase. The
+ response buffer does not really use it since it is immediately
+ forwarded to the client.
msg.sol : start of line. Points to the beginning of the current header line
while parsing headers. It is cleared to zero in the BODY state,
@@ -97,7 +102,8 @@ msg.eol : end of line. Points to the CRLF or LF of the current header line
states nor by forwarding.
The beginning of the message headers can always be found this way even after
-headers have been forwarded :
+headers or data have been forwarded, provided that everything is still present
+in the buffer :
headers = buf.p + msg->sov - msg->eoh - msg->eol
diff --git a/include/types/proto_http.h b/include/types/proto_http.h
index 12e1141..c53c7fd 100644
--- a/include/types/proto_http.h
+++ b/include/types/proto_http.h
@@ -329,7 +329,8 @@ enum {
* message or a response message.
*
* The values there are a little bit obscure, because their meaning can change
- * during the parsing :
+ * during the parsing. Please read carefully doc/internal/body-parsing.txt if
+ * you need to manipulate them. Quick reminder :
*
* - eoh (End of Headers) : relative offset in the buffer of first byte that
* is not part of a completely processed header.
@@ -344,9 +345,9 @@ enum {
* - sov (start of value) : Before HTTP_MSG_BODY, points to the value of
* the header being parsed. Starting from
* HTTP_MSG_BODY, will point to the start of the
- * body (relative to buffer's origin), or to data
- * following a chunk size. Thus <sov> bytes of
- * headers will have to be sent only once.
+ * body (relative to buffer's origin). It can be
+ * negative when forwarding data. It stops growing
+ * once data start to leave the buffer.
*
* - next (parse pointer) : next relative byte to be parsed. Always points
* to a byte matching the current state.
@@ -372,7 +373,7 @@ struct http_msg {
/* 6 bytes unused here */
struct channel *chn; /* pointer to the channel transporting the message */
unsigned int next; /* pointer to next byte to parse, relative to buf->p */
- unsigned int sov; /* current header: start of value */
+ int sov; /* current header: start of value ; data: start of body */
unsigned int eoh; /* End Of Headers, relative to buffer */
unsigned int sol; /* start of current line during parsing otherwise zero */
unsigned int eol; /* end of line */
diff --git a/src/proto_http.c b/src/proto_http.c
index 4a862b0..94afed7 100644
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -5315,7 +5315,7 @@ int http_request_forward_body(struct session *s, struct channel *req, int an_bit
* an "Expect: 100-continue" header.
*/
- if (msg->sov) {
+ if (msg->sov > 0) {
/* we have msg->sov which points to the first byte of message
* body, and req->buf.p still points to the beginning of the
* message. We forward the headers now, as we don't need them
@@ -5429,6 +5429,8 @@ int http_request_forward_body(struct session *s, struct channel *req, int an_bit
* such as last chunk of data or trailers.
*/
b_adv(req->buf, msg->next);
+ if (unlikely(!(s->rep->flags & CF_READ_ATTACHED)))
+ msg->sov -= msg->next;
msg->next = 0;
/* for keep-alive we don't want to forward closes on DONE */
@@ -5479,6 +5481,9 @@ int http_request_forward_body(struct session *s, struct channel *req, int an_bit
missing_data:
/* we may have some pending data starting at req->buf->p */
b_adv(req->buf, msg->next);
+ if (unlikely(!(s->rep->flags & CF_READ_ATTACHED)))
+ msg->sov -= msg->next + MIN(msg->chunk_len, req->buf->i);
+
msg->next = 0;
msg->chunk_len -= channel_forward(req, msg->chunk_len);
@@ -6493,7 +6498,7 @@ int http_response_forward_body(struct session *s, struct channel *res, int an_bi
/* in most states, we should abort in case of early close */
channel_auto_close(res);
- if (msg->sov) {
+ if (msg->sov > 0) {
/* we have msg->sov which points to the first byte of message
* body, and res->buf.p still points to the beginning of the
* message. We forward the headers now, as we don't need them
--
1.8.5.5

48
net/haproxy/patches/0025-DOC-remove-references-to-CPU-native-in-the-README.patch Normal file
View file

@ -0,0 +1,48 @@
From 94fb38fbb77e664e4f41343257a26ae5bab40d1d Mon Sep 17 00:00:00 2001
From: Willy Tarreau <w@1wt.eu>
Date: Thu, 10 Jul 2014 20:24:25 +0200
Subject: [PATCH 25/25] DOC: remove references to CPU=native in the README
Certain compilers running in virtualized environments may produce code
that the same processor cannot execute with -march=native, either because
of hypervisor bugs reporting wrong CPU features, or because of compiler
bugs forgetting to check CPU features. So better stop recommending this
combination so that users don't get trapped anymore.
(cherry picked from commit 817dad50b02d1a82d495dfea4eab9e3a91127391)
---
README | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/README b/README
index 0ef0179..e2b8570 100644
--- a/README
+++ b/README
@@ -53,8 +53,9 @@ one of the following choices to the CPU variable :
- i686 for intel PentiumPro, Pentium 2 and above, AMD Athlon
- i586 for intel Pentium, AMD K6, VIA C3.
- ultrasparc : Sun UltraSparc I/II/III/IV processor
- - native : use the build machine's specific processor optimizations
- - generic : any other processor or no specific optimization. (default)
+ - native : use the build machine's specific processor optimizations. Use with
+ extreme care, and never in virtualized environments (known to break).
+ - generic : any other processor or no CPU-specific optimization. (default)
Alternatively, you may just set the CPU_CFLAGS value to the optimal GCC options
for your platform.
@@ -132,11 +133,11 @@ And I build it this way on OpenBSD or FreeBSD :
And on a classic Linux with SSL and ZLIB support (eg: Red Hat 5.x) :
- $ make TARGET=linux26 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
+ $ make TARGET=linux26 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
And on a recent Linux >= 2.6.28 with SSL and ZLIB support :
- $ make TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
+ $ make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
In order to build a 32-bit binary on an x86_64 Linux system with SSL support
without support for compression but when OpenSSL requires ZLIB anyway :
--
1.8.5.5