Apache Subversion 1.7.0-rc3 Released

September 15, 2011 by admin · Leave a Comment 

Apache Subversion 1.7.0-rc3 has been released.

Apache Subversion 1.7 is a superset of all previous Subversion releases, and is as of the time of its release considered the current “best” release. Any feature or bugfix in 1.0.x through 1.6.x is also in 1.7, but 1.7 contains features and bugfixes not present in any earlier release. The new features will eventually be documented in a 1.7 version of the free Subversion book (svnbook.red-bean.com).

This page describes only major changes. For a complete list of changes, see the 1.7 section of the CHANGES file.

Apache HTTP Server 2.2.20 Released

August 31, 2011 by admin · Leave a Comment 

Apache HTTP Server 2.2.20 has been released.

The Apache HTTP Server Project is proud to announce the release of version 2.2.20 of the Apache HTTP Server (“httpd”). This version is principally a security (CVE-2011-3192) and bugfix release .

This version of httpd is a major release of the stable branch, and represents the best available version of Apache HTTP Server. New features include Smart Filtering, Improved Caching, AJP Proxy, Proxy Load Balancing, Graceful Shutdown support, Large File Support, the Event MPM, and refactored Authentication/Authorization.

Download | New Features in httpd 2.2 | ChangeLog for 2.2.20 | Complete ChangeLog for 2.2

Apache Traffic Server 3.1.0 stable Released

August 30, 2011 by admin · Leave a Comment 

Apache Traffic Server 3.1.0 stable has been released.

Apache Traffic Server™ is fast, scalable and extensible HTTP/1.1 compliant caching proxy server. Formerly a commercial product, Yahoo! donated it to the Apache Foundation, and it is now an Apache TLP. Here’s a Traffic Server overview.

Apache Traffic Server 3.0.1 was released on July 19, 2011. [PGP] [MD5] [SHA1]

Bugs

  • [TS-842] – remove uninstall target from build system
  • [TS-868] – build fails with –as-needed.
  • [TS-860] – Built in error for host not found looks like Internet Explorer’s
  • [TS-875] – TSFetchRestpGet(), TSFetchPageResptGet() and TSFetchUrl() have incorrect asserts
  • [TS-834] – InactivityCop::check_inactivity crash
  • [TS-826] – TSHttpTxnErrorBodySet() can leak memory.
  • [TS-833] – Continuation::handleEvent deadbeef fix. This fixes most of the dns related crashes
  • [TS-828] – Various memory leaks and uninitialized values.
  • [TS-839] – Build problems when specifying lmza location.
  • [TS-840] – Regression checks fail (again) due to faulty assert use.
  • [TS-847] – Bad timeout when using CONNECT method.

The full change log is located in /trafficserver/traffic/branches/3.0.x/CHANGES.

Apache POI 3.8 beta4 Released

August 30, 2011 by admin · Leave a Comment 

Apache POI 3.8 beta4 has been released.

The Apache POI team is pleased to announce the release of 3.8 beta 4. This includes a large number of bug fixes and enhancements.

A full list of changes is available in the change log. People interested should also follow the dev mailing list to track further progress.

See the downloads page for more details.

Apache James 3.0-beta3 Released

August 29, 2011 by admin · Leave a Comment 

Apache James 3.0-beta3 has been released.

The Apache James Project delivers a rich set of open source modules and libraries, written in Java, related to Internet mail communication which build into an advanced enterprise mail server.

If you’re interested in contributing to the James project, please subscribe to the James mailing lists.

        * What's new in 3.0-beta3 for end users
          - Numerous IMAP bug fixes (better client support, memory improvement, NIO and IO support...)
          - Support for IMAP IDLE (RFC 2177, server transmit updates to the client in real time)
          - Support for IMAP User Flags
          - Support for IMAP WITHIN Extensions (RFC 5032)
          - Mailbox Tooling to copy from a persistence implementation to another implementation
          - Telnet Management has been removed in favor of JMX with client shell
          - More metrics counters available via JMX
          - Better debug logging on protocols
          - JPA validated against more databases (among others Oracle)
          - Multiple address and port configurations per protocol
          - POP3 is now operational (was buggy in 3.0-M2)
          - Upgrade tool from James 2.3 is available
          - Better logging on protocols with adjustable level
          - Full mailet package must be specified
          - Composite Matchers
          - Better debug logging on protocols
          - Mailing list functionality has been removed
          - More documentation on web site for configuration,...
          - Java 1.6 mandatory
          - ... and much more, see details on https://issues.apache.org/jira/secure/ReleaseNote.jspa?in&version=12316850&styleName=Html&projectId=10411
        * What's new in 3.0-beta3 for developers
          - Less maven modules
          - Maven 3.0.2 required to build
          - Upgrade to latest frameworks versions (netty, activemq, jackrabbit...)
          - Code reports generation via 'mvn site -P site-reports' maven profile
          - Corrections further to findbugs,... reports
          - Code formatting
          - ... and much more, see details on https://issues.apache.org/jira/secure/ReleaseNote.jspa?in&version=12316850&styleName=Html&projectId=10411

        * Quick Start  http://james.apache.org/server/3/quick-start.html

FreeBSD + Apache + nginx proxy + vhosts

August 24, 2011 by admin · Leave a Comment 

Here is a excellent tutorial show you about FreeBSD + Apache + nginx proxy + vhosts:

Luckily I have not been the victim of a slowloris attack but that is no reason not to take action. So when I was setting up a webserver solution based on Apache, MySQL and PHP5 I might as well add a caching frontend for Apache. This howto guide assumes you have basic knowledge of FreeBSD and Apache already installed.

The idea is to let Apache carry the heavy lifting on the backend and let nginx serve the files to the visitors. The frees up Apache to focus on generating the pages and data needed then send it to nginx and thus being ready for the next job. nginx will then just server the files to the visitor at the speed they have, using a lot less resources.

Apache Jackrabbit 2.2.8 Released

August 24, 2011 by admin · Leave a Comment 

Apache Jackrabbit 2.2.8 has been released.

The Apache Jackrabbit™ content repository is a fully conforming implementation of the Content Repository for Java Technology API (JCR, specified in JSR 170 and 283).

A content repository is a hierarchical content store with support for structured and unstructured content, full text search, versioning, transactions, observation, and more.

Apache Jackrabbit 2.2.8 is patch release that contains fixes and improvements over previous 2.2.x releases. See the downloads page for more details.

Apache OpenNLP 1.5.1 Released

August 23, 2011 by admin · Leave a Comment 

Apache OpenNLP 1.5.1 has been released.

OpenNLP is an organizational center for open source projects related to natural language processing. Its primary role is to encourage and facilitate the collaboration of researchers and developers on such projects.

OpenNLP also hosts a variety of java-based NLP tools which perform sentence detection, tokenization, pos-tagging, chunking and parsing, named-entity detection, and coreference using the OpenNLP Maxent machine learning package.

This release contains a number of improvements and bug fixes compared to the last SourceForge 1.5.0 release. Most notably, the wiki documentation was converted to docbook, the F-Measure precision was fixed, perceptron bugs were fixed, CoNLL 2003 training format support was added, chunker evaluation support was added, the chunker now supports Portuguese Bosque AD format, and the chunker was refactored

Apache mod_proxy_ajp Module

August 22, 2011 by admin · Leave a Comment 

Apache mod_proxy_ajp Module:

This module requires the service of mod_proxy. It provides support for the Apache JServ Protocol version 1.3 (hereafter AJP13).

Thus, in order to get the ability of handling AJP13 protocol, mod_proxy and mod_proxy_ajp have to be present in the server.

Environment Variables

Environment variables whose names have the prefix AJP_ are forwarded to the origin server as AJP request attributes (with the AJP_ prefix removed from the name of the key).

top

Overview of the protocol

The AJP13 protocol is packet-oriented. A binary format was presumably chosen over the more readable plain text for reasons of performance. The web server communicates with the servlet container over TCP connections. To cut down on the expensive process of socket creation, the web server will attempt to maintain persistent TCP connections to the servlet container, and to reuse a connection for multiple request/response cycles.

Once a connection is assigned to a particular request, it will not be used for any others until the request-handling cycle has terminated. In other words, requests are not multiplexed over connections. This makes for much simpler code at either end of the connection, although it does cause more connections to be open at once.

Once the web server has opened a connection to the servlet container, the connection can be in one of the following states:

  • Idle
    No request is being handled over this connection.
  • Assigned
    The connecton is handling a specific request.

Once a connection is assigned to handle a particular request, the basic request informaton (e.g. HTTP headers, etc) is sent over the connection in a highly condensed form (e.g. common strings are encoded as integers). Details of that format are below in Request Packet Structure. If there is a body to the request (content-length > 0), that is sent in a separate packet immediately after.

At this point, the servlet container is presumably ready to start processing the request. As it does so, it can send the following messages back to the web server:

  • SEND_HEADERS
    Send a set of headers back to the browser.
  • SEND_BODY_CHUNK
    Send a chunk of body data back to the browser.
  • GET_BODY_CHUNK
    Get further data from the request if it hasn’t all been transferred yet. This is necessary because the packets have a fixed maximum size and arbitrary amounts of data can be included the body of a request (for uploaded files, for example). (Note: this is unrelated to HTTP chunked tranfer).
  • END_RESPONSE
    Finish the request-handling cycle.

Each message is accompanied by a differently formatted packet of data. See Response Packet Structures below for details.

top

Basic Packet Structure

There is a bit of an XDR heritage to this protocol, but it differs in lots of ways (no 4 byte alignment, for example).

Byte order: I am not clear about the endian-ness of the individual bytes. I’m guessing the bytes are little-endian, because that’s what XDR specifies, and I’m guessing that sys/socket library is magically making that so (on the C side). If anyone with a better knowledge of socket calls can step in, that would be great.

There are four data types in the protocol: bytes, booleans, integers and strings.

Byte
A single byte.
Boolean
A single byte, 1 = true, 0 = false. Using other non-zero values as true (i.e. C-style) may work in some places, but it won’t in others.
Integer
A number in the range of 0 to 2^16 (32768). Stored in 2 bytes with the high-order byte first.
String
A variable-sized string (length bounded by 2^16). Encoded with the length packed into two bytes first, followed by the string (including the terminating ‘\0′). Note that the encoded length does not include the trailing ‘\0′ — it is like strlen. This is a touch confusing on the Java side, which is littered with odd autoincrement statements to skip over these terminators. I believe the reason this was done was to allow the C code to be extra efficient when reading strings which the servlet container is sending back — with the terminating \0 character, the C code can pass around references into a single buffer, without copying. if the \0 was missing, the C code would have to copy things out in order to get its notion of a string.

Packet Size

According to much of the code, the max packet size is 8 * 1024 bytes (8K). The actual length of the packet is encoded in the header.

Packet Headers

Packets sent from the server to the container begin with 0x1234. Packets sent from the container to the server begin with AB (that’s the ASCII code for A followed by the ASCII code for B). After those first two bytes, there is an integer (encoded as above) with the length of the payload. Although this might suggest that the maximum payload could be as large as 2^16, in fact, the code sets the maximum to be 8K.

Packet Format (Server->Container)
Byte 0 1 2 3 4…(n+3)
Contents 0×12 0×34 Data Length (n) Data
Packet Format (Container->Server)
Byte 0 1 2 3 4…(n+3)
Contents A B Data Length (n) Data

For most packets, the first byte of the payload encodes the type of message. The exception is for request body packets sent from the server to the container — they are sent with a standard packet header ( 0x1234 and then length of the packet), but without any prefix code after that.

The web server can send the following messages to the servlet container:

Code Type of Packet Meaning
2 Forward Request Begin the request-processing cycle with the following data
7 Shutdown The web server asks the container to shut itself down.
8 Ping The web server asks the container to take control (secure login phase).
10 CPing The web server asks the container to respond quickly with a CPong.
none Data Size (2 bytes) and corresponding body data.

To ensure some basic security, the container will only actually do the Shutdown if the request comes from the same machine on which it’s hosted.

The first Data packet is send immediatly after the Forward Request by the web server.

The servlet container can send the following types of messages to the webserver:

Code Type of Packet Meaning
3 Send Body Chunk Send a chunk of the body from the servlet container to the web server (and presumably, onto the browser).
4 Send Headers Send the response headers from the servlet container to the web server (and presumably, onto the browser).
5 End Response Marks the end of the response (and thus the request-handling cycle).
6 Get Body Chunk Get further data from the request if it hasn’t all been transferred yet.
9 CPong Reply The reply to a CPing request

Each of the above messages has a different internal structure, detailed below.

top

Request Packet Structure

For messages from the server to the container of type Forward Request:

AJP13_FORWARD_REQUEST :=
    prefix_code      (byte) 0x02 = JK_AJP13_FORWARD_REQUEST
    method           (byte)
    protocol         (string)
    req_uri          (string)
    remote_addr      (string)
    remote_host      (string)
    server_name      (string)
    server_port      (integer)
    is_ssl           (boolean)
    num_headers      (integer)
    request_headers *(req_header_name req_header_value)
    attributes      *(attribut_name attribute_value)
    request_terminator (byte) OxFF

The request_headers have the following structure:

req_header_name :=
    sc_req_header_name | (string)  [see below for how this is parsed]

sc_req_header_name := 0xA0xx (integer)

req_header_value := (string)

The attributes are optional and have the following structure:

attribute_name := sc_a_name | (sc_a_req_attribute string)

attribute_value := (string)

Not that the all-important header is content-length, because it determines whether or not the container looks for another packet immediately.

Detailed description of the elements of Forward Request

Request prefix

For all requests, this will be 2. See above for details on other Prefix codes.

Method

The HTTP method, encoded as a single byte:

Command Name Code
OPTIONS 1
GET 2
HEAD 3
POST 4
PUT 5
DELETE 6
TRACE 7
PROPFIND 8
PROPPATCH 9
MKCOL 10
COPY 11
MOVE 12
LOCK 13
UNLOCK 14
ACL 15
REPORT 16
VERSION-CONTROL 17
CHECKIN 18
CHECKOUT 19
UNCHECKOUT 20
SEARCH 21
MKWORKSPACE 22
UPDATE 23
LABEL 24
MERGE 25
BASELINE_CONTROL 26
MKACTIVITY 27

Later version of ajp13, will transport additional methods, even if they are not in this list.

protocol, req_uri, remote_addr, remote_host, server_name, server_port, is_ssl

These are all fairly self-explanatory. Each of these is required, and will be sent for every request.

Headers

The structure of request_headers is the following: First, the number of headers num_headers is encoded. Then, a series of header name req_header_name / value req_header_value pairs follows. Common header names are encoded as integers, to save space. If the header name is not in the list of basic headers, it is encoded normally (as a string, with prefixed length). The list of common headers sc_req_header_nameand their codes is as follows (all are case-sensitive):

Name Code value Code name
accept 0xA001 SC_REQ_ACCEPT
accept-charset 0xA002 SC_REQ_ACCEPT_CHARSET
accept-encoding 0xA003 SC_REQ_ACCEPT_ENCODING
accept-language 0xA004 SC_REQ_ACCEPT_LANGUAGE
authorization 0xA005 SC_REQ_AUTHORIZATION
connection 0xA006 SC_REQ_CONNECTION
content-type 0xA007 SC_REQ_CONTENT_TYPE
content-length 0xA008 SC_REQ_CONTENT_LENGTH
cookie 0xA009 SC_REQ_COOKIE
cookie2 0xA00A SC_REQ_COOKIE2
host 0xA00B SC_REQ_HOST
pragma 0xA00C SC_REQ_PRAGMA
referer 0xA00D SC_REQ_REFERER
user-agent 0xA00E SC_REQ_USER_AGENT

The Java code that reads this grabs the first two-byte integer and if it sees an '0xA0' in the most significant byte, it uses the integer in the second byte as an index into an array of header names. If the first byte is not 0xA0, it assumes that the two-byte integer is the length of a string, which is then read in.

This works on the assumption that no header names will have length greater than 0x9999 (==0xA000 - 1), which is perfectly reasonable, though somewhat arbitrary.

Note:

The content-length header is extremely important. If it is present and non-zero, the container assumes that the request has a body (a POST request, for example), and immediately reads a separate packet off the input stream to get that body.

Attributes

The attributes prefixed with a ? (e.g. ?context) are all optional. For each, there is a single byte code to indicate the type of attribute, and then its value (string or integer). They can be sent in any order (though the C code always sends them in the order listed below). A special terminating code is sent to signal the end of the list of optional attributes. The list of byte codes is:

Information Code Value Type Of Value Note
?context 0×01 - Not currently implemented
?servlet_path 0×02 - Not currently implemented
?remote_user 0×03 String  
?auth_type 0×04 String  
?query_string 0×05 String  
?jvm_route 0×06 String  
?ssl_cert 0×07 String  
?ssl_cipher 0×08 String  
?ssl_session 0×09 String  
?req_attribute 0x0A String Name (the name of the attribute follows)
?ssl_key_size 0x0B Integer  
are_done 0xFF - request_terminator

The context and servlet_path are not currently set by the C code, and most of the Java code completely ignores whatever is sent over for those fields (and some of it will actually break if a string is sent along after one of those codes). I don’t know if this is a bug or an unimplemented feature or just vestigial code, but it’s missing from both sides of the connection.

The remote_user and auth_type presumably refer to HTTP-level authentication, and communicate the remote user’s username and the type of authentication used to establish their identity (e.g. Basic, Digest).

The query_string, ssl_cert, ssl_cipher, and ssl_session refer to the corresponding pieces of HTTP and HTTPS.

The jvm_route, is used to support sticky sessions — associating a user’s sesson with a particular Tomcat instance in the presence of multiple, load-balancing servers.

Beyond this list of basic attributes, any number of other attributes can be sent via the req_attribute code 0x0A. A pair of strings to represent the attribute name and value are sent immediately after each instance of that code. Environment values are passed in via this method.

Finally, after all the attributes have been sent, the attribute terminator, 0xFF, is sent. This signals both the end of the list of attributes and also then end of the Request Packet.

top

Response Packet Structure

for messages which the container can send back to the server.

AJP13_SEND_BODY_CHUNK :=
  prefix_code   3
  chunk_length  (integer)
  chunk        *(byte)
  chunk_terminator (byte) Ox00

AJP13_SEND_HEADERS :=
  prefix_code       4
  http_status_code  (integer)
  http_status_msg   (string)
  num_headers       (integer)
  response_headers *(res_header_name header_value)

res_header_name :=
    sc_res_header_name | (string)   [see below for how this is parsed]

sc_res_header_name := 0xA0 (byte)

header_value := (string)

AJP13_END_RESPONSE :=
  prefix_code       5
  reuse             (boolean)

AJP13_GET_BODY_CHUNK :=
  prefix_code       6
  requested_length  (integer)

Details:

Send Body Chunk

The chunk is basically binary data, and is sent directly back to the browser.

Send Headers

The status code and message are the usual HTTP things (e.g. 200 and OK). The response header names are encoded the same way the request header names are. See header_encoding above for details about how the codes are distinguished from the strings.
The codes for common headers are:

Name Code value
Content-Type 0xA001
Content-Language 0xA002
Content-Length 0xA003
Date 0xA004
Last-Modified 0xA005
Location 0xA006
Set-Cookie 0xA007
Set-Cookie2 0xA008
Servlet-Engine 0xA009
Status 0xA00A
WWW-Authenticate 0xA00B

After the code or the string header name, the header value is immediately encoded.

End Response

Signals the end of this request-handling cycle. If the reuse flag is true (==1), this TCP connection can now be used to handle new incoming requests. If reuse is false (anything other than 1 in the actual C code), the connection should be closed.

Get Body Chunk

The container asks for more data from the request (If the body was too large to fit in the first packet sent over or when the request is chuncked). The server will send a body packet back with an amount of data which is the minimum of the request_length, the maximum send body size (8186 (8 Kbytes - 6)), and the number of bytes actually left to send from the request body.
If there is no more data in the body (i.e. the servlet container is trying to read past the end of the body), the server will send back an empty packet, which is a body packet with a payload length of 0. (0x12,0x34,0x00,0x00)

Apache mod_backhand Howto

August 22, 2011 by admin · Leave a Comment 

Apache mod_backhand Howto:

mod_backhand is project that allows seamless redirection of HTTP requests from one web server to another. This redirection can be used to target machines with under-utilized resources, thus providing fine-grained, per-request load balancing of web requests.

Backhand is a project that was initiated in class at The Johns Hopkins University in The Department of Computer Science. After a simple proof of concept, written in C++, the project was pursued as a graduate qualifier project.

A proof of concept was realized, but a deployable product was needed. In order for public use, testing and, ultimately, acceptance, we wished to provide drop-in functionality in the most popular web server on the net.

mod_backhand was born. This drop-in module for the Apache Web Server provides a turnkey solution for intra-cluster redirection of HTTP on a per-request basis. The redirection is based on various system resources available within the cluster. This service is provided seamlessly, preserving remote host information for access/authentication purposes. As with the methodology of Apache, the redirection mechanism can be enabled or disable for individual directories.

System resources are announced by the module to the cluster via ethernet broadcasts and/or IP multicasts. These resources are made available to the decision making algorithms that drive mod_backhand.

Next Page »