Lighttpd mod_mysql_vhost Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_mysql_vhost Module:

With MySQL-based vhosting you can store the path to a given host’s document root in a MySQL database.

Note: Keep in mind that only one vhost module should be active at a time. Don’t mix mod_simple_vhost with mod_mysql_vhost.

Options

Example:

mysql-vhost.db             = "lighttpd"
mysql-vhost.user           = "lighttpd"
mysql-vhost.pass           = "secret"
mysql-vhost.sock           = "/var/run/mysqld/mysqld.sock"
mysql-vhost.sql            = "SELECT docroot FROM domains WHERE domain='?';"
mysql-vhost.hostname           = "localhost"
mysql-vhost.port           = 3306

If specified, mysql-vhost.hostname overrides mysql-vhost.sock.

MySQL setup:

GRANT SELECT ON lighttpd.* TO lighttpd@localhost IDENTIFIED BY 'secret';
CREATE DATABASE lighttpd;
USE lighttpd;
CREATE TABLE domains (
  domain varchar(64) not null primary key,
  docroot varchar(128) not null
);
INSERT INTO domains VALUES ('host.dom.ain','/http/host.dom.ain/');

Wildcard subdomains with same docroot

Insert domain names like ‘%.example.com’ into the sql table, and use the following query:

mysql-vhost.sql            = "SELECT docroot FROM domains WHERE '?' like domain;"

Per-vhost configuration (not using the module)

You can obviously create a lighttpd config from a database (load it via the include_shell command in the lighttpd config);
but there is no point in using the same table again with mod_mysql_vhost – just use server.document-root.

Your script should output something like that for every vhost:

$HTTP["host"] == "<DOMAIN_FROM_DATABASE>" {
  server.document-root = "<DOCROOT_FROM_DATABASE>"
  <CONFIG_FROM_DATABASE>
}

Please note that you should not allow your users to modify the config which gets inserted here, or they just may use include_shell with “rm -rf /” – which gets probably executed with root.

And you have to restart your webserver every time you change something.

Lighttpd mod_secdownload Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_secdownload Module:

There are multiple ways to handle secured download mechanisms:

  1. use the webserver and the internal HTTP authentication
  2. use the application to authenticate and send the file
    through the application

Both ways have limitations:

Webserver:

  • + fast download
  • + no additional system load
  • – inflexible authentication handling

Application:

  • + integrated into the overall layout
  • + very flexible permission management
  • – the download occupies an application thread/process

A simple way to combine the two ways could be:

1. app authenticates user and checks permissions to
download the file.
2. app redirects user to the file accessable by the webserver
for further downloading.
3. the webserver transfers the file to the user.

As the webserver doesn’t know anything about the permissions
used in the app, the resulting URL would be available to every
user who knows the URL.

mod_secdownload removes this problem by introducing a way to
authenticate a URL for a specified time. The application has
to generate a token and a timestamp which are checked by the
webserver before it allows the file to be downloaded by the
webserver.

The generated URL has to have the format:

<uri-prefix>/<token>/<timestamp-in-hex>/<rel-path>
which looks like "yourserver.com/bf32df9cdb54894b22e09d0ed87326fc/435cc8cc/secure.tar.gz" 

<token> is an MD5 of

1. a secret string (user supplied)
2. <rel-path> (starts with /)
3. <timestamp-in-hex>

As you can see, the token is not bound to the user at all. The
only limiting factor is the timestamp which is used to
invalidate the URL after a given timeout (secdownload.timeout).

Important

Be sure to choose a another secret than the one used in the
examples, as this is the only part of the token that is not
known to the user.

Ensure that the token is also in hexadecimal. Depending on
the programming language you use, there might be no extra
step for this. For instance, in PHP, the MD5 function
returns the Hex value of the digest. If, however, you use a
language such as Java or Python, the extra step of converting
the digest into Hex is needed (see the Python example below).

If the user tries to fake the URL by choosing a random token,
status 403 ‘Forbidden’ will be sent out.

If the timeout is reached, status 410 ‘Gone’ will be
sent. This used to be 408 ‘Request Timeout’ in earlier versions.

If token and timeout are valid, the <rel-path> is appended to
the configured (secdownload.document-root) and passed to the
normal internal file transfer functionality. This might lead to
status 200 or 404.

Options

  secdownload.secret        = <string>
  secdownload.document-root = <string>
  secdownload.uri-prefix    = <string>  (default: /)
  secdownload.timeout       = <short>   (default: 60 seconds)

Examples

Your application has to generate the correct URLs.

PHP Example

  <?php

  $secret = "verysecret";
  $uri_prefix = "/dl/";

  # filename
  # please note file name starts with "/"
  $f = "/secret-file.txt";

  # current timestamp
  $t = time();

  $t_hex = sprintf("%08x", $t);
  $m = md5($secret.$f.$t_hex);

  # generate link
  printf('<a href="%s%s/%s%s">%s</a>',
         $uri_prefix, $m, $t_hex, $f, $f);
  ?>

Ruby On Rails example, used in the context of a helper

  def gen_sec_link(rel_path)
    rel_path.sub!(/^([^\/])/,'/\1')     # Make sure it had a leading slash
    s_secret = 'verysecret'             # Secret string
    uri_prefix = '/dl/'                 # Arbitrary download prefix
    timestamp = "%08x" % Time.now.to_i  # Timestamp, to hex
    token = MD5::md5(s_secret + rel_path + timestamp).to_s    # Token Creation
    '%s%s/%s%s' % [uri_prefix, token, timestamp, rel_path]   # Return the properly formatted string
  end

So in a view or helper:

<%= link_to "Private Image", gen_sec_link("path/from/download-area/someimage.img") %>

Perl Example

#!/usr/bin/perl

use CGI;
use Digest::MD5 qw(md5 md5_hex md5_base64);

  $secret = "verysecret";
  $uri_prefix = "/static/";

  #filename
  $f = "/$request";

  #timestamp
  $t = time();

  $hex = sprintf("%08x", $t);
  $m = md5_hex($secret.$f.$t_hex);

  #link
  printf('<a href="%s%s/%s%s">%s</a>', $uri_prefix, $m, $t_hex, $f, $f);

Python example, usable with Django or any other Python web framework

  def gen_sec_link(rel_path):
      import time, hashlib
      secret = 'verysecret'
      uri_prefix = '/dl/'
      hextime = "%08x" % time.time()
      token = hashlib.md5(secret + rel_path + hextime).hexdigest()
      return '%s%s/%s%s' % (uri_prefix, token, hextime, rel_path)

Note: When using Django with non-ASCII file names (hashlib in Python 2.x does not handle non-ASCII strings properly, throwing the UnicodeDecodeError exception, unless bytestrings encoded in UTF-8 are supplied instead of the default unicode objects):

  def gen_securelink(rel_path):
      import time, hashlib
      from django.utils.http import urlquote
      rel_path = '/%s/%s' % (self.series.directory, self.filename)
      secret = "flylight"
      uri_prefix = "http://media.ongoing.ru/download/"
      hextime = "%08x" % time.time()
      token = hashlib.md5((secret + rel_path + hextime).encode('utf-8')).hexdigest()
      return '%s%s/%s%s' % (uri_prefix, token, hextime, urlquote(rel_path))

(urlquote() can be replaced by urllib methods, of course.)

C# Example

//import library

using System.Security.Cryptography;
using System.Text;

//function :
    public string GetHash(string hashMe) // function get MD5
    {
        MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
        UTF7Encoding encoder = new UTF7Encoding();
        Byte[] encStringBytes;
        encStringBytes = encoder.GetBytes(hashMe);
        encStringBytes = md5.ComputeHash(encStringBytes);
        string strHex = string.Empty;
        foreach (byte b in encStringBytes)
            strHex += String.Format("{0:x2}", b);
        return strHex;
    }
    public string GetCurrentEpochTimeInHex() // function get current epoch time in HEX
    {
        DateTime EpochTime = new DateTime(1970, 1, 1, 0, 0, 0);
        TimeSpan diff = DateTime.UtcNow - EpochTime;
        return ((long)diff.TotalSeconds).ToString("x");
    }
    public string GenerateSecureLink(string rel_path) // ex : rel_path = "/secret-file.txt";
    {
        string t_hex = GetCurrentEpochTimeInHex();
        string serect = "verysecret";
        string uri_prefix = "/dl/";
        string m = GetHash(serect + rel_path + t_hex);
        return String.Format("{0}{1}/{2}{3}", uri_prefix, m, t_hex, rel_path);
    }

Reference make MD5 in C# : http://ok-cool.com/posts/read/125-php-md5-not-the-same-as-net-md5/

Webserver

The server has to be configured in the same way. The URI prefix and
secret have to match: ::

  server.modules = ( ..., "mod_secdownload", ... )

  secdownload.secret          = "verysecret"
  secdownload.document-root   = "/home/www/servers/download-area/"
  secdownload.uri-prefix      = "/dl/"
  secdownload.timeout         = 10

Lighttpd mod_mem_cache Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_mem_cache Module:

Serving several thousands small files for lighttpd is quite challenging. Operating System has to do a lot of random access, which increase io-wait. mod_mem_cache is the plugin which stores content of files in memory for faster serving. It’s more configurable than OS buffers (at least, linux buffers), in some cases yields better performance.

For example, you can configure it to cache files smaller than 5 kb, just cache files with a specific mime type or different strategies to free memory when the configured limit has been reached.

Installation

This module is a 3rd party module, is not included in the official distribution. You need to download it from here:
For lighttpd 1.4.13: [http://blog.quehy.com/tag/mod_mem_cache mod_mem_cache]
For lighttpd 1.4.15: [http://bergstroem.nu/lighttpd-1.4.15.mod_mem_cache.v1.2.patch] (unsupported, straight port from 1.4.13 patch)
For lighttpd 1.4.19: [http://blog.quehy.com/archives/197.html], Patch File for Freebsd Ports [http://trac.lighttpd.net/trac/attachment/wiki/Docs/ModMemCache/patch-mod_mem_cache.patch]
For lighttpd 1.4.xx and svn Updated Patchfile (works with rev. 2639): [http://redmine.lighttpd.net/attachments/1026/lighttpd-1.4.x-svn-mod_mem_cache.patch]
For lighttpd 1.5.x: [http://groups.google.com/group/mod_cache/files], patches for non-stable versions

Important
mod_mem_cache and mod_cml (–with-lua) do not play nicely together since mod_mem_cache modifies how lighttpd handles cachable items. There probably won’t be a patch for this since mod_cml_lua is deprecated.

Options

See [http://blog.quehy.com/archives/171.html]

mem-cache.filetypes
   content-type arrays which want to put into memory.

mem-cache.filetypes=("text/css")
   Default: not set, cache all files

mem-cache.enable
   enable or disable mod_mem_cache.

mem-cache.enable = "disable"
    $HTTP["host"] == "images.example.org" {
      mem-cache.enable = "enable"
    }

   Default: enable

mem-cache.max-memory
    maxium memory in Mbytes mod-mem-cache can use.

    Default: 512 (means 512 ''Megabytes'').

mem-cache.max-file-size
    maxium file size in Kbytes of single file to cache in memory.

    Default: 512 (means 512 ''Kilobytes'').

mem-cache.lru-remove-count
    number of cached memory items to remove when used memory reached maxmemory by LRU algorthim

    Default: 200.

mem-cache.expire-time
    memory cache's expire time in minutes. 

mem-cache.expire-time = 1440 # 1 day

    Default: 0 (means to check file update for every request)

mem-cache.slru-thresold
    Adjusts slru threshold (against hit counter)

    Default: 0 (disabled)

Example

  • mem-cache.filetypes = (“application/x-javascript”, “text/css”, “text/html”, “text/javascript”) # Cache javascript, css, html file in memory
  • mem-cache.max-memory = 1024 # Use 1G memory cache
  • mem-cache.max-file-size = 1024 # Cache maxium 1M-size file

Lighttpd mod_uploadprogress Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_uploadprogress Module:

upload-progress.progress-url

string, empty by default; use “/progress” (or something else) to enable the module

upload-progress.remove-timeout

integer, default: 60 seconds

upload-progress.debug

boolean, default: “disable”

Config Example

upload-progress.progress-url = "/progress"
upload-progress.remove-timeout = 30
upload-progress.debug = "enable"

Warning

This module will not work if you are using more than one worker (“server.max-worker”:Server.max-workerDetails)

Description

This module can be used to track the progress of a current upload.

When a file upload is done, the current received bytes are tracked
with this module and can be retrieved via the progress-url and the
tracking key. This key has to be added to the upload url or via
header.

In combination with server.max-request-size, usually the browser
will display an error page containing information that the connection
has been terminated. This can not be replaced with a custom error
page, the browsers aren’t displaying them.

The uploadprogress module tracks this errors now. If the upload
is done in one window (or an iframe which will be hidden on form submit),
another iframe can display a custom uploadprogress page which can
also fetch the status 413 via json if this happen. If 413 is retrieved,
a custom error message (File is too big) can be displayed.

The first examples can be found at
http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back

The mentioned multi-frame example will be added in the future.

How to use it

Generate a progress-id (32 hex digits – just use a md5 hash); send this progress-id in the upload request and in all progress requests, either as “X-Progress-ID” header or as “X-Progress-ID” query string parameter.

JSON Response

The returned json may contain:

  • state
    current state of upload
    values = starting, error, done, uploading
  • status
    http error status
    values = 413
  • size
    size of request
  • received
    bytes received by lighttpd yet

Example 1

Three html-files are appended as example, based on Jan’s work mentioned
at the link above. You need to add the tracking_id (an example how this is
done with a single page upload and javascript is also provided at Jan’s blog).

In the iframe version, you can generate the tracking_id via php or other
server-side scripting.

Warning, the files are not working out of the box, you have to change things!

Downloads:

Example 2

Theres a new example (using YUI framework) here:
http://redmine.lighttpd.net/attachments/399/index.2.html

mod_uploadprogress for lighttpd-1.4 branch

I’ve backported mod_uploadprogress so you may use it together with lighttpd-1.4 branch if interested. You may download the patch at http://labs.logic.cz/patches/lighttpd/1.4.x/

Don’t expect exactly the same behavior as this version returns JSON object for better integration with frameworks like Dojo. I’ve also tweaked the key names and some values returned.

We’re using this patch for about half a year and it seems stable. Enjoy!

index.html – example 1 – index.html (2.2 kB) stbuehler, 2009-02-17 09:50

status.html – example 1 – status.html (4.2 kB) stbuehler, 2009-02-17 09:50

upload.html – example 1 – upload.html (1.2 kB) stbuehler, 2009-02-17 09:50

Lighttpd mod_scgi Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_scgi Module:

The SCGI module is heavily based on the FastCGI when it comes to configuration.

Only the internal protocol between server and client has been replaced.

Please check the documentation of the FastCGI module for more information.

History

Added in lighttpd 1.3.14 as it was really simple to do.

C/C++ SCGI on named socket

Up to lightty 1.4.19 there is a bug in that the pre-opened socket handle is not on descriptor 0 (as it is for FastCGI FCGI_LISTENSOCK_FILENO) but any later file descriptor.

Since 1.4.20 you can take over your testfastcgi program and replace FCGX_Accept() by a standard Unix accept(LISTENSOCK_FILENO, &addr, &addrlen) – then pthread_create/fork for your handler.

The SCGI parameter block at the start of the socket stream is easy to read and to decode – no third-party library is required anymore.

Just write() to the accept’ed socket handle to return the result page to the lightty webserver and close() the accept’ed handle to signal that the request has finished.

Lighttpd mod_rrdtool Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_rrdtool Module:

RRD is a system to store and display time-series data (i.e. network bandwidth, machine-room temperature, server load average).

Warning

  • This module will not work if you are using more than one worker (“server.max-worker”:Server.max-workerDetails)

Options

rrdtool.binary

path to the rrdtool binary

rrdtool.binary = "/usr/bin/rrdtool"

rrdtool.db-name

filename of the rrd-database. Make sure that <rrdtool.db-name> doesn’t exist before the first run, as lighttpd has to create the DB for you.

rrdtool.db-name = "/var/www/lighttpd.rrd"

Generating Graphs

Four things must be done to generate graphs:

Alternative: Instead of the last three steps (2,3,4), just can use lightygraph.cgi to automatically generate the graphs as needed — on the fly!

  1. Enable the mod_rrdtool in lighttpd.conf (as above)
  2. Create/Edit the script provided below (as it generates the graphs).
    I call it rrdtool.sh, make sure you have eXecute permissions on it (chmod +x)
  3. Create an HTML page to output the images (provided at the end bottom)
  4. Run the script and add to cron.
    0,20,40 * * * * nice -n 10 /etc/lighttpd/rrdtool.sh >& /dev/null

    Note: Ubuntu fiesty and later remove the & from >& above for cron to run. /bin/sh is linked to /bin/dash and expects a file descriptor rather than a path to send the output to. (bad fd error otherwise)

rrdtool.sh

#!/bin/sh

RRDTOOL=/usr/bin/rrdtool
OUTDIR=/var/www/www.example.com/rrd/
INFILE=/var/www/lighttpd.rrd
OUTPRE=lighttpd-traffic
WIDTH=400
HEIGHT=100

DISP="-v bytes --title TrafficWebserver \
        DEF:binraw=$INFILE:InOctets:AVERAGE \
        DEF:binmaxraw=$INFILE:InOctets:MAX \
        DEF:binminraw=$INFILE:InOctets:MIN \
        DEF:bout=$INFILE:OutOctets:AVERAGE \
        DEF:boutmax=$INFILE:OutOctets:MAX \
        DEF:boutmin=$INFILE:OutOctets:MIN \
        CDEF:bin=binraw,-1,* \
        CDEF:binmax=binmaxraw,-1,* \
        CDEF:binmin=binminraw,-1,* \
        CDEF:binminmax=binmaxraw,binminraw,- \
        CDEF:boutminmax=boutmax,boutmin,- \
        AREA:binmin#ffffff: \
        STACK:binmax#f00000: \
        LINE1:binmin#a0a0a0: \
        LINE1:binmax#a0a0a0: \
        LINE2:bin#efb71d:incoming \
        GPRINT:bin:MIN:%.2lf \
        GPRINT:bin:AVERAGE:%.2lf \
        GPRINT:bin:MAX:%.2lf \
        AREA:boutmin#ffffff: \
        STACK:boutminmax#00f000: \
        LINE1:boutmin#a0a0a0: \
        LINE1:boutmax#a0a0a0: \
        LINE2:bout#a0a735:outgoing \
        GPRINT:bout:MIN:%.2lf \
        GPRINT:bout:AVERAGE:%.2lf \
        GPRINT:bout:MAX:%.2lf \
        " 

$RRDTOOL graph $OUTDIR/$OUTPRE-hour.png -a PNG --start -14400 $DISP -w $WIDTH -h $HEIGHT
$RRDTOOL graph $OUTDIR/$OUTPRE-day.png -a PNG --start -86400 $DISP -w $WIDTH -h $HEIGHT
$RRDTOOL graph $OUTDIR/$OUTPRE-month.png -a PNG --start -2592000 $DISP -w $WIDTH -h $HEIGHT

OUTPRE=lighttpd-requests

DISP="-v req --title RequestsperSecond -u 1 \
        DEF:req=$INFILE:Requests:AVERAGE \
        DEF:reqmax=$INFILE:Requests:MAX \
        DEF:reqmin=$INFILE:Requests:MIN \
        CDEF:reqminmax=reqmax,reqmin,- \
        AREA:reqmin#ffffff: \
        STACK:reqminmax#00f000: \
        LINE1:reqmin#a0a0a0: \
        LINE1:reqmax#a0a0a0: \
        LINE2:req#00a735:requests" 

$RRDTOOL graph $OUTDIR/$OUTPRE-hour.png -a PNG --start -14400 $DISP -w $WIDTH -h $HEIGHT
$RRDTOOL graph $OUTDIR/$OUTPRE-day.png -a PNG --start -86400 $DISP -w $WIDTH -h $HEIGHT
$RRDTOOL graph $OUTDIR/$OUTPRE-month.png -a PNG --start -2592000 $DISP -w $WIDTH -h $HEIGHT

Webpage that displays the graphics

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
        <title>Lighttpd traffic &amp; requests</title>
        <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
        <meta http-equiv="content-style-type" content="text/css">
        <style type="text/css">
<!--
        div { text-align:center; }
        img { width:693px; height:431px; }
-->
        </style>
</head>

<body>
    <div>
        <h2>Lighttpd Traffic</h2>
        <img src="lighttpd-traffic-hour.png"   alt="graph1"><br>
        <img src="lighttpd-traffic-day.png"    alt="graph2"><br>
        <img src="lighttpd-traffic-month.png"  alt="graph3"><br>
    </div>
    <div>
        <h2>Lighttpd Requests</h2>
        <img src="lighttpd-requests-hour.png"  alt="graph4"><br>
        <img src="lighttpd-requests-day.png"   alt="graph5"><br>
        <img src="lighttpd-requests-month.png" alt="graph6"><br>
    </div>
  </body>
</html>

Lighttpd mod_webdav Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_webdav Module:

The WebDAV module is a very minimalistic implementation of RFC 2518. Minimalistic means that not all operations are implemented yet.

So far we have:

  • PROPFIND
  • OPTIONS
  • MKCOL
  • DELETE
  • PUT
  • LOCK (experimental)

and the usual GET, POST, HEAD from HTTP/1.1.

If you want to setup a WebDAV repository with authentication, make sure you are loading mod_webdav `before` mod_auth.

So far, mounting an open-access WebDAV resource into Windows XP (Network Places), Mac OS X (Finder) and Linux (Konqueror) works. Mounting an authenticated WebDAV resource works in Mac OS X and Linux. The basic litmus tests are passed.

Installation

mod_webdav is part of the 1.4.x distribution.

PUT, DELETE

If you are only looking for PUT and DELETE support, the basic mod_webdav works for you. No special compile-time options or libraries are required. PUT and DELETE are already part of the HTTP/1.1 spec, but are mostly used in combination with WebDAV.

PROPFIND, PROPPATCH

PROPFIND and PROPPATCH are used to do a directory-listing and to attach properties to a file-object.

As this involves parsing of XML-data we need libxml2. To make PROPPATCH work you need sqlite3 to store the properties in a separate location:

  • libxml2FC4:
    libxml2 libxml2-devel
  • sqlite3FC4:
    sqlite sqlite-devel

To compile mod_webdav with property-support you have to specify: ::

$ configure --with-webdav-props

LOCK, UNLOCK

LOCK and UNLOCK are necessary to get mounting of file-systems working. This involves generate unique lock-tokens. We use libuuid for this job.

  • libuuidFC4:
    e2fsprogs e2fsprogs-devel

LOCK with includes parsing XML again, so have to provide the dependencies of the previous section and libuuid: ::

$ ./configure --with-webdav-props --with-webdav-locks

Options

webdav.activate
If you load the webdav module, the WebDAV functionality has to be enabled for the directories you want to provide to the user.

Default: disable

webdav.is-readonly
Only allow reading methods (GET, PROPFIND, OPTIONS) on WebDAV resources.

Default: writable

webdav.sqlite-db-name
The full path to the file you would like to use as your db file. This is required for webdav props and locks.

Default: &lt;empty&gt;

Examples

To enable WebDAV for the /dav directory, you wrap your webdav options in a conditional. You have to use the regex like below as you want to match the directory /dav and everything below it, but not e.g. /davos. ::

  $HTTP["url"] =~ "^/dav($|/)" {
    webdav.activate = "enable"
    webdav.is-readonly = "enable"
    webdav.sqlite-db-name = "/var/run/lighttpd/lighttpd.webdav_lock.db"
  }

If you would like LOCK support ::

  $HTTP["url"] =~ "^/dav($|/)" {
    webdav.activate = "enable"
    webdav.is-readonly = "disable"
    webdav.sqlite-db-name = "/var/run/lighttpd/lighttpd.webdav_lock.db"
  }

note : The WebDAV directory should be a child of the document root. So if you want to add a server.document-root option to the /dav configuration, the path should be a child of the server.document-root (Otherwise move and copy function will not work correctly). (Take a look in mod_webdav.c line 1830)

litmus tests

We use http://webdav.org/neon/litmus/ (version 0.10.5) to verify that we support at least the basic of the WebDAV spec.

For now we are far away from 100%.

  • we don’t handle If: … headers
  • we handle shared locks as exclusive locks
  • the failing basic.8 is new in 1.4.13 as we stripped off the fragments for clean URLs

Handle the If: header requires a parser as it contains logical expressions. To give you an overview what works and what isn’t: ::

  -> running 'basic':
   8. delete_fragment....... WARNING: DELETE removed collection resource with Request-URI including fragment; unsafe
      ...................... pass (with 1 warning)
  <- summary for `basic': of 15 tests run: 15 passed, 0 failed. 100.0%
  -> 1 warning was issued.
  -> running `copymove':
   4. copy_overwrite........ WARNING: COPY to existing resource didn't give 204
      ...................... pass (with 1 warning)
   8. move.................. WARNING: MOVE to existing collection resource didn't give 204
      ...................... pass (with 1 warning
  <- summary for `copymove': of 12 tests run: 12 passed, 0 failed. 100.0%
  -> 2 warnings were issued.
  -> running `props':
  <- summary for `props': of 26 tests run: 26 passed, 0 failed. 100.0%
  -> running `locks':
  16. fail_cond_put......... FAIL (conditional PUT with invalid lock-token should fail: 200 OK)
  18. cond_put_corrupt_token FAIL (conditional PUT with invalid lock-token should fail: 200 OK)
  20. fail_complex_cond_put. FAIL (PUT with complex bogus conditional should fail with 412: 200 OK)
  22. fail_cond_put_unlocked FAIL (conditional PUT with invalid lock-token should fail: 200 OK)
  27. double_sharedlock..... FAIL (shared LOCK on locked resource: 423 Locked)
  34. notowner_modify....... FAIL (DELETE of locked resource should fail)
  <- summary for `locks': of 39 tests run: 33 passed, 6 failed. 84.6%

Lighttpd mod_status Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_status Module:

The server status module generates the status overview of the webserver. The
information covers:

  • uptime
  • average throughput
  • current throughput
  • active connections and their state

We need to load the module first.

server.modules = ( ..., "mod_status", ... )

By default the status page is disabled to hide internal information from
unauthorized users.

status.status-url = "/server-status"

If you want to open the status page just for users from the local network
cover it in a conditional.

$HTTP["remoteip"] == "10.0.0.0/8" {
    status.status-url = "/server-status"
  }

Or require authorization:

auth.require = ( "/server-status" =>
    ( "realm" ... ) )

Careful : mod_auth must be loaded before mod_status so it works.

Please note that when using the server.max-worker directive, the stati of the
childs are not combined yet, so you’re going to see different stats with each
request.

Output Format

By default a nice looking HTML page is generated. If you append ?auto to the
status-url you can get a text version which is simpler to parse.

Total Accesses: 1234
  Total kBytes: 1043
  Uptime: 1234
  BusyServers: 123

Total Accesses is the number of handled requests, kBytes the overall outgoing
traffic, Uptime the uptime in seconds and BusyServers the number of currently
active connections.

The naming is kept compatible to Apache even if we have another concept and
don’t start new servers for each connection.

Server Statistics

The server statistics is a set of counters used to track the different stati in several modules.
For example :trac:`wiki:Docs:ModFastCGI mod_fastcgi` is using it to track the usage of the backends:

fastcgi.active-requests: 0
  fastcgi.backend.fcgi-php.0.connected: 10127
  fastcgi.backend.fcgi-php.0.died: 0
  fastcgi.backend.fcgi-php.0.disabled: 0
  fastcgi.backend.fcgi-php.0.load: 0
  fastcgi.backend.fcgi-php.0.overloaded: 0
  fastcgi.backend.fcgi-php.1.connected: 93855
  fastcgi.backend.fcgi-php.1.died: 0
  fastcgi.backend.fcgi-php.1.disabled: 0
  fastcgi.backend.fcgi-php.1.load: 0
  fastcgi.backend.fcgi-php.1.overloaded: 0
  fastcgi.backend.fcgi-php.load: 1
  fastcgi.requests: 399355
fastcgi.requests

is a counter of the requests since the start of the server.

fastcgi.active-requests

counter of the currently active requests in the module

fastcgi.backend.*.*.load

the active requests by backend. If the load is above the number
of backend processes/threads for this backend the requests will
have to wait until a this backend is working again.

Options

status.status-url

relative URL which is used to retrieve the status-page

Default: unset

Example:
status.status-url = “/server-status”

status.enable-sort

add JavaScript which allows client-side sorting for the connection overview

Default: enable

status.config-url

relative URL for the config page which displays the loaded modules

Default: unset

Example:

status.config-url = "/server-config"
status.statistics-url

relative URL for a plain-text page containing the internal statistics

Default: unset

Example:

status.statistics-url = "/server-statistics"

Note: if you receive a 204 (no content / your browser doesn’t seems to do anything) when requesting ”/server-statistics” via your browser is because there’s no counter data. Feels weird, but it’s 100% correct (at least, for the HTTP standard)

Lighttpd SSL Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd SSL Module:

lighttpd supports SSLv2 (disabled from 1.4.21 and 1.5 and onwards) and SSLv3 if it is compiled against openssl.

How to install SSL

To use SSL you must have ssl compiled into lighty. You must first have
openssl-devel installed and openssl installed as well. On Fedora or
Centos you may use yum to install this by running this command:

yum install openssl*

And type yes when it asks for a confirmation of what you would like to install.

Once installed please download the lighttpd tarball from http://www.lighttpd.net/download and extract it with

tar zxvf lighttpd-1.4.20.tar.gz

Enter into the extracted folder and compile with these configuration flags: ::

–with-openssl –with-openssl-libs=/usr/lib

Remember to change _ –with-openssl-libs= _ to the folder where your openssl libraries are installed into.

Once compiled, run make and make install. If lighty has successfully compiled SSL the command

lighttpd -v

Should display (Keep in mind that this new lighty version now has (ssl) after lightys name)

lighttpd-1.4.11 (ssl) – a light and fast webserver
Build-Date: Sep 1 2006 19:09:15

Remember if you used the RPM packages to install lighty, the init.d scripts will point to the wrong binary of lighty than the one you just compiled. The location of where you compiled lighty should be displayed near the end of make install. Once the location of the binary is found please edit the /etc/init.d/lighttpd script and change what is defined in the lighttpd=”/usr/sbin/lighttpd” to your new lighty location.

Configuration

option description
ssl.engine enable/disable ssl engine
ssl.pemfile path to the PEM file for SSL support (Should contain both the private key and the certificate)
ssl.ca-file path to the CA file for support of chained certificates
ssl.dh-file path to the PEM file for Diffie-Hellman key agreement protocol (lighttpd >= 1.4.29 only)
ssl.ec-curve defines the set of elliptic-curve-cryptography domain parameters known as a “named curve” (lighttpd >= 1.4.29 only)
ssl.use-sslv2 enable/disable use of SSL version 2 (lighttpd < 1.4.21 only, newer version don’t support SSLv2)
ssl.use-sslv3 enable/disable use of SSL version 3 (lighttpd >= 1.4.29 only)
ssl.cipher-list Configure the allowed SSL ciphers
ssl.verifyclient.activate enable/disable client verification
ssl.verifyclient.enforce enable/disable enforcing client verification
ssl.verifyclient.depth certificate depth for client verification
ssl.verifyclient.exportcert enable/disable client certificate export to env:SSL_CLIENT_CERT
ssl.verifyclient.username client certificate entity to export as env:REMOTE_USER (eg. SSL_CLIENT_S_DN_emailAddress, SSL_CLIENT_S_DN_UID, etc.)

Details

To enable SSL you have to provide a valid certificate and have to enable the SSL engine. To make lighttpd SSL-only, simply put the following in your main config (you probably need to set the server port to 443 as well).

ssl.engine = "enable"
 ssl.pemfile = "/path/to/server.pem"

To enable SSL in addition to normal HTTP, put the ssl.engine configuration in a socket conditional block:

$SERVER["socket"] == ":443" {
     ssl.engine                  = "enable"
     ssl.pemfile                 = "/etc/lighttpd/ssl/www.example.org.pem"
 }

This simply enables SSL on port 443. It seems that lighttpd also automatically starts listening on port 443 this way. Like this, all of the other configuration applies to both HTTP and HTTPS. This means that the same pages will be available on both HTTP and HTTPS. If you want to serve different sites, you can change the document root inside the socket conditional:

$SERVER["socket"] == ":443" {
    server.document-root        = "/www/servers/www.example.org/secure/"
 }

When you are using lighttpd 1.4.19 or later, you can also use the scheme conditional to distinguish between secure and normal requests. Note that you can’t use the scheme conditional around ssl.engine above, since lighttpd needs to know on what port to enable SSL.

$HTTP["scheme"] == "https" {
    server.document-root        = "/www/servers/www.example.org/secure/"
 }

If you have a .crt and a .key file, cat them together into a single PEM file (the order isn’t strictly important):

$ cat host.key host.crt > host.pem

ssl.cipher-list is a list of ciphers that you want (or don’t want) to use when talking SSL. For a list of ciphers and how to include/exclude them, see sections “CIPHER LIST FORMAT” and “CIPHER STRINGS” on the man-page for ciphers.

Permissions

Be careful to keep your .pem file private! Lighttpd reads all pemfiles at startup, before dropping privileges. It is therefore best to make the pem file owned by root and readable by root only:

$ chown root:root /etc/lighttpd/ssl/example.org.pem
  $ chmod 400 /etc/lighttpd/ssl/example.org.pem

Chained certificates

Some certificate authorities use chained certificates. This means that your webserver certificate is not signed by the CA root certificate directly, but by an intermediate certificate (which is in turn signed by the root CA). Since browser typically only ship the root certificates and not all intermediate certificates, there is no way for a browser to verify your chained certificate if you don’t supply the browser with the intermediate certificate.

Using the ssl.ca-file configuration variable, you can tell lighttpd about these intermediate certificates. You can put multiple certificates in a single file, if needed. For example:

ssl.ca-file = "/etc/lighttpd/ssl/ca.crt"

Troubleshooting

  • Please note that certificates’s files are opened before the server goes chrooted (if it’s have to). So these paths are system wide.
  • On server side, ssldump is your friend: ssldump -i your_network_interface_goes_here port 443
  • On client side, openssl will help you: openssl s_client -connect your_server_name_goes_here:443

SSL passwords

  • If you set a password on your SSL certificate, then each time lighttpd starts you will be requested to enter it manually. ie: “Enter PEM pass phrase:”. To prevent this, remove the password from your private key file. At present there is no configuration option to enable storage of SSL certificate passwords in the lighttpd config file (but it wouldn’t make much sense, since having the password stored on disk is not more secure than having no password at all).

Diffie-Hellman and Elliptic-Curve Diffie-Hellman parameters

Diffie-Hellman and Elliptic-Curve Diffie-Hellman key agreement protocols will be supported in lighttpd 1.4.29. By default, Diffie-Hellman and Elliptic-Curve Diffie-Hellman key agreement protocols use, respectively, the 1024-bit MODP Group with 160-bit prime order subgroup from RFC 5114 and “prime256v1″ (also known as “secp256r1″) elliptic curve from RFC 4492. The Elliptic-Curve Diffie-Hellman key agreement protocol is supported in OpenSSL from 0.9.8f version onwards. For maximum interoperability, OpenSSL only supports the “named curves” from RFC 4492.

Using the ssl.dh-file and ssl.ec-curve configuration variables, you can define your own set of Diffie-Hellman domain parameters. For example:

ssl.dh-file = "/etc/lighttpd/ssl/dh2048.pem"
ssl.ec-curve = "secp384r1"

A set of Diffie-Hellman domain parameters can be generated like this (see OpenSSL documentation for help):

$ openssl dhparam -out dh2048.pem -outform PEM -2 2048

The set of supported elliptic-curves can be obtained like this (see OpenSSL documentation for help):

$ openssl ecparam -list_curves

SSL on multiple domains

A traditional problem with SSL in combination with name based virtual hosting has been that the SSL connection setup happens before the HTTP request. So at the moment lighttpd needs to send its certificate to the client, it does not know yet which domain the client will be requesting. This means it can only supply the default certificate (and use the corresponding key for encryption) and effectively, SSL can only be enabled for that default domain. There are a number of solutions to this problem, with varying levels of support.

One IP address per domain

This is the classic solution, which requires a separate IP address for each domain. Since Lighttpd knows the ip address that the client connected to before the SSL negotiation, it can simply select the right certificate to use based on the IP address. The most obvious downside to this approach is that it requires multiple addresses, which are not always available and can be expensive. This approach is currently the most reliable, since it works with all clients.

To do this in lighttpd, use the socket conditional:

$SERVER["socket"] == "10.0.0.1:443" {
        ssl.pemfile = "/etc/lighttpd/ssl/www.example.org.pem"
    }
$SERVER["socket"] == "10.0.0.2:443" {
        ssl.pemfile = "/etc/lighttpd/ssl/mail.example.org.pem"
    }

UCC / SAN Certificates

It is possible to put multiple names in a single certificate. These certificates are usually referred to as UCC (Unified Communications Certificates) certificates, or SAN (Subject Alternative Name) certificates. These certificates use the SAN property to store multiple domain names in a single certificate. This allows lighttpd to always use the same certificate, which is valid for all the domains it serves.

The main disadvantage of this approach is that you will have a single certificate with a lot of domain names in it and the certificate needs to change whenever you add a new domain name. When serving sites for different parties, this can be a problem.

According to digicert, SAN certificates are supported by nearly all browsers (except for some mobile browsers) and should be pretty safe to use.

Using a SAN certificate requires no special configuration for Lighttpd.

Wildcard certificates

Certificates support the use of wildcards, which can be useful to support multiple subdomains. For example, one can get a certificate for the *.example.org domain to support all of the subdomains of the example.org domain.

This approach of course does not scale well to multiple different domains. Again according to digicert, wildcard certificates should be supported by virtually all clients.

Server Name Indication (SNI)

An extension to the SSL / TLS protocol that has gained support recently is Server Name Indication. This addition is simple and elegant: During the SSL negotiation, the client sends the name of the host it wants to contact to the server, which can then use this information to select the correct certificate.

This is of course the best and most fitting solution to the problem. Lighttpd supports SNI since 1.4.24, if openssl 0.9.8f or newer is available.

The big downside of this approach is the limited browser support. The newest versions of the major browsers support SNI, but in particular Internet Explorer on Windows XP seems to lack support and support with mobile browsers is unclear.

To use SNI with lighttpd, simply put additional ssl.pemfile configuration directives inside host conditionals. It seems a default ssl.pemfile is still required, though.

$HTTP["host"] == "www.example.org" {
        ssl.pemfile = "/etc/lighttpd/www.example.org.pem"
    }
$HTTP["host"] == "mail.example.org" {
        ssl.pemfile = "/etc/lighttpd/mail.example.org.pem"
    }

Other issues

Self-Signed Certificates

A self-signed SSL certificate can be generated like this: ::

$ openssl req -new -x509 \
    -keyout server.pem -out server.pem \
    -days 365 -nodes

PCI DSS compliance

Matthew Glubb wrote:

I should clarify the reason for this work. From September 19th, all major online vendors taking card payments will be required to comply with the Payment Card Industry (PCI) Data Security Standard. Smaller vendors may self-certify but they will be more liable if fraud is committed.

Part of this standard is the disabling of SSLv2 and the removal of support for ciphers that have a key length of less than 128 bits. For this reason, I believe that the default SSL configuration for lighttpd should reflect this standard.

Since 1.4.12 you can use: ::

ssl.use-sslv2 = "disable"   -   (No longer needed as of 1.4.21 and 1.5 this is disabled by default.)
  ssl.cipher-list = "..."

to disable SSLv2 and set a cipher-list. Make sure this settings are in the same block than ssl.engine = “enable”, this will not work as global options.

cipher-list accepts a string containing the ciphers you would like to accept separated by whitespace. A list of strings will not work.

Matthew also provide a list of possible ciphers:

Hope it can be of use. Next time I’ll submit a proper svn patch but I was in a hurry! Which might explain my somewhat short list of supported ciphers. I’ve since done some research and this list of supported ciphers is much more comprehensive. It supports all ciphers >= 128 bit key lengths for SSL v3.0, TLS v1.0, and AES cipher suites from RFC3268, extending TLS v1.0 (these seem to be the ones used by recent browsers, not included in the original list): ::

RC4-SHA
    RC4-MD5
    ADH-RC4-MD5
    EDH-RSA-DES-CBC3-SHA
    EDH-DSS-DES-CBC3-SHA
    DES-CBC3-SHA
    ADH-DES-CBC3-SHA
    DES-CBC3-MD5
    AES128-SHA
    AES256-SHA
    DH-DSS-AES128-SHA
    DH-DSS-AES256-SHA
    DH-RSA-AES128-SHA
    DH-RSA-AES256-SHA
    DHE-DSS-AES128-SHA
    DHE-DSS-AES256-SHA
    DHE-RSA-AES128-SHA
    DHE-RSA-AES256-SHA

[FYI] bb says: the requirements permit use of SSLv2 as long as it isn’t the only version available. for example, having SSLv2, SSLv3, and TLSv1 enabled is PCI compliant. SSLv2 should still be disabled, however, both because it is weak and because you can expect the PCI requirements to become more stringent in the future. here is my recommended cipher list, in order, which also happens to be PCI compliant: ::

DHE-RSA-AES256-SHA
    DHE-RSA-AES128-SHA
    EDH-RSA-DES-CBC3-SHA
    AES256-SHA
    AES128-SHA
    DES-CBC3-SHA
    DES-CBC3-MD5
    RC4-SHA
    RC4-MD5
  • Note: you are forgetting the CAMELLIA cipher (Firefox 3 default) which going by it’s wikipedia page is roughly equivalent to AES. For a good list do `openssl ciphers HIGH`

PCI DSS compliance

How to enable that the server requests a SSL client certificate?

HTTPS detection in PHP

Some PHP scripts try to detect HTTPS by checking if $_SERVER['HTTPS'] equals ‘on’. To allow that, you can try this: ::

server.modules = (
        "mod_setenv",
    )
    $SERVER["socket"] == "0.0.0.0:443" {
        ssl.engine             = "enable"
        ssl.pemfile            = "/etc/lighttpd/server.pem"
        ssl.use-sslv2          = "disable"
        setenv.add-environment = (
            "HTTPS" => "on"
        )
    }

Lighttpd mod_extforward Module

August 22, 2011 by admin · Leave a Comment 

Lighttpd mod_extforward Module:

This module will extract the client’s “real” IP from X-Forwarded-For header which is added by Squid or other proxies. It might be useful for servers behind reverse proxy servers.

Options

extforward.headers
Sets headers to search for finding the originl addresses.
Default: empty, results in searching for “X-Forwarded-For” and “Forwarded-For”

Example (for use with a Zeus ZXTM loadbalancer):

    extforward.headers = ("X-Cluster-Client-Ip")

 

extforward.forwarder
Sets trust level of proxy IP’s.
Default: empty

Example of usage:

    extforward.forwarder = ("10.0.0.232" => "trust")

 

will translate ip addresses coming from 10.0.0.232 to real ip addresses extracted from X-Forwarded-For: HTTP request header.

Important note:

The effect of this module is variable on $HTTP["remoteip"] directives and other module’s remote ip dependent actions.
Things done by modules before we change the remoteip or after we reset it will match on the proxy’s IP.
Things done in between these two moments will match on the real client’s IP.
The moment things are done by a module depends on in which hook it does things and within the same hook
on whether they are before/after us in the module loading order
(order in the server.modules directive in the config file).

Tested behaviours:

mod_access: Will match on the real client.

mod_accesslog: In order to see the “real” ip address in access log, you’ll have to load mod_extforward after mod_accesslog, like this:

    server.modules  = (
       .....
       "mod_accesslog",
       "mod_extforward"
    )

Samples

Trust proxy 10.0.0.232 and 10.0.0.233

  extforward.forwarder = (
     "10.0.0.232" => "trust",
     "10.0.0.233" => "trust"
  )

Trust all proxies (NOT RECOMMENDED!)

  extforward.forwarder = ( "all" => "trust")

Note that “all” has precedence over specific entries, so “all except” setups will not work.

Installation

mod_extforward is included in lighttpd 1.4.14 and later

Next Page »