[Ericsson AB]

4 HTTP server

4.1 Introduction

The HTTP server, also refered to as httpd, handles HTTP requests as described in RFC 2616 with a few exceptions such as gateway and proxy functionality. (The same is true for servers written by NCSA and others.) The server supports ipv6 as long as the underlying mechanisms also do so.

The server implements numerous features such as SSL (Secure Sockets Layer), ESI (Erlang Scripting Interface), CGI (Common Gateway Interface), User Authentication(using Mnesia, dets or plain text database), Common Logfile Format (with or without disk_log(3) support), URL Aliasing, Action Mappings, Directory Listings and SSI (Server-Side Includes).

The configuration of the server is done using Apache-style configuration directives..

Allmost all server functionality has been implemented using an especially crafted server API, it is described in the Erlang Web Server API. This API can be used to advantage by all who wants to enhance the server core functionality, for example custom logging and authentication.

4.2 Basic Configuration

It is possible to start a number of Web servers in an embedded system using the services config parameter from an application config file. A minimal application config file (from now on referred to as inets.config) starting two HTTP servers typically looks as follows: 

      [{inets,
        [{services, [{httpd, "/var/tmp/server_root/conf/8888.conf"},
                     {httpd, "/var/tmp/server_root/conf/8080.conf"}]
         }
        ]
       }
      ].

or: 

      [{inets,
        [{services, [{httpd, [{file,"/var/tmp/server_root/conf/8888.conf"}]},
                     {httpd, [{file,"/var/tmp/server_root/conf/8080.conf"}]}]
         }
        ]
       }
      ].

According to the new syntax which allows more functionality in the configuration. The possible options here are a customer configurable request accept timeout, the default value is 15000 milliseconds, and some trace functionality to debug the http server. The syntax must match the following grammar: 

     httpd_service() -> {httpd, httpd()}
     httpd()         -> [httpd_config()] | file()
     httpd_config()  -> {file, file()} | 
                        {debug, debug()} |
                        {accept_timeout, integer()}
     debug()         -> disable | [debug_options()]
     debug_options() -> {all_functions, modules()} | 
                        {exported_functions, modules()} |
                        {disable, modules()}
     modules()       -> [atom()]

{file, file()} corresponds to the functionality of the old version.

{debug, debug()} is the new trace option. It can trace on all functions or only exported functions on choosen modules.

{accept_timeout, integer()} sets the wanted timeout value for the server to set up a request connection.

A server config file is specified for each HTTP server to be started. The server config file syntax and semantics is described in the run time configuration section.

An easy way to test the setup of inets webservers can be done by copying the example server root (UNIX: $INETS_ROOT/examples/server_root/conf/, Windows: %INETS_ROOT%\examples\server_root\conf\) to a specific installation directory (/var/tmp/server_root/conf in this example). Then manualy start the Erlang node, using inets.config. 

      $ erl -config ./inets
      Erlang (BEAM) emulator version 4.9
      
      Eshell V4.9 (abort with ^G) 1> application:start(inets).
      ok

Now there should be two HTTP servers started listening on the ports 8888 and 8080. You can test it by using any browser or the inets HTTP client requesting the urls: http://localhost:8888 and http://localhost:8080

4.3 Server Runtime Configuration

All functionality in the server can be configured using Apache-style configuration directives stored in a configuration file. A minimal configuration file could look something like: 

      ServerName web.server.net
      ServerRoot /var/tmp/server_root
      DocumentRoot /var/tmp/server_root/htdocs

E.i the syntax is Directive followed by a withspace followed by the value of the directive followed by a new line.

The available directives are described in the section Server Configuration Directives.

4.4 Server Configuration Directives

4.4.1 Mandantory Directives

4.4.2 Communication Directives

4.4.3 Limit Directives

4.4.4 Administrative Directives

4.4.5 SSL Directives

4.4.6 URL Aliasing

4.4.7 CGI Directives

4.4.8 ESI Directives

4.4.9 Auth Directives

4.4.10 Htacess Authentication Directives

4.4.11 Auth Filter Directives

4.4.12 Logging Directives

4.4.13 Disk Log Directives

4.5 Mime Type Configuration

Files delivered to the client are MIME typed according to RFC 1590. File suffixes are mapped to MIME types before file delivery.

The mapping between file suffixes and MIME types are specified in the mime.types file. The mime.types reside within the conf directory of the ServerRoot. MIME types may be added as required to the mime.types file and the DefaultType config directive can be used to specify a default mime type. An example of a very small mime.types file: 

    # MIME type                 Extension  
    text/html                   html htm
    text/plain                  asc txt

4.6 Htaccess - User Configurable Authentication.

If users of the webserver needs to manage authentication of webpages that are local to their user and do not have server administrative privileges. They can use the per-directory runtime configurable user-authentication scheme that Inets calls htaccess. It works the following way:

4.6.1 Access Files Directives

In every directory under the DocumentRoot or under an Alias a user can place an access-file. An access-file is a plain text file that specify the restrictions that shall be considered before the webserver answer to a request. If there are more than one access-file in the path to the requested asset, the directives in the access-file in the directory nearest the asset will be used.

4.7 Dynamic Web Pages

The Inets HTTP server provides two ways of creating dynamic web pages, each with its own advantages and disadvantages.

First there are CGI-scripts that can be written in any programming language. CGI-scripts are standardized and supported by most webservers. The drawback with CGI-scripts is that they are resource intensive because of their design. CGI requires the server to fork a new OS process for each executable it needs to start.

Second there are ESI-functions that provide a tight and efficient interface to the execution of Erlang functions, this interface on the other hand is Inets specific.

4.7.1 The Common Gateway Interface (CGI) Version 1.1, RFC 3875.

The mod_cgi module makes it possible to execute CGI scripts in the server. A file that matches the definition of a ScriptAlias config directive is treated as a CGI script. A CGI script is executed by the server and it's output is returned to the client.

The CGI Script response comprises a message-header and a message-body, separated by a blank line. The message-header contains one or more header fields. The body may be empty. Example: 

"Content-Type:text/plain\nAccept-Ranges:none\n\nsome very
        plain text"

The server will interpret the cgi-headers and most of them will be transformed into HTTP headers and sent back to the client together with the body.

Support for CGI-1.1 is implemented in accordance with the RFC 3875.

4.7.2 Erlang Server Interface (ESI)

The erlang server interface is implemented by the module mod_esi.

4.7.2.1 ERL Scheme

The erl scheme is designed to mimic plain CGI, but without the extra overhead. An URL which calls an Erlang erl function has the following syntax (regular expression): 

          http://your.server.org/***/Module[:/]Function(?QueryString|/PathInfo)

*** above depends on how the ErlScriptAlias config directive has been used

The module (Module) referred to must be found in the code path, and it must define a function (Function) with an arity of two or three. It is preferable to implement a funtion with arity three as it permitts you to send chunks of the webpage beeing generated to the client during the generation phase instead of first generating the whole web page and then sending it to the client. The option to implement a function with arity two is only keept for backwardcompatibilty reasons. See mod_esi(3) for implementation details of the esi callback function.

4.7.2.2 EVAL Scheme

The eval scheme is straight-forward and does not mimic the behavior of plain CGI. An URL which calls an Erlang eval function has the following syntax: 

http://your.server.org/***/Mod:Func(Arg1,...,ArgN)

*** above depends on how the ErlScriptAlias config directive has been used

The module (Mod) referred to must be found in the code path, and data returned by the function (Func) is passed back to the client. Data returned from the function must furthermore take the form as specified in the CGI specification. See mod_esi(3) for implementation details of the esi callback function.

Note!

The eval scheme can seriously threaten the integrity of the Erlang node housing a Web server, for example: 

http://your.server.org/eval?httpd_example:print(atom_to_list(apply(erlang,halt,[])))

which effectively will close down the Erlang node, that is use the erl scheme instead, until this security breach has been fixed.

Today there are no good way of solving this problem and therefore Eval Scheme may be removed in future release of Inets.

4.8 Logging

There are three types of logs supported. Transfer logs, security logs and error logs. The de-facto standard Common Logfile Format is used for the transfer and security logging. There are numerous statistics programs available to analyze Common Logfile Format. The Common Logfile Format looks as follows:

remotehost rfc931 authuser [date] "request" status bytes

remotehost
Remote hostname
rfc931
The client's remote username (RFC 931).
authuser
The username with which the user authenticated himself.
[date]
Date and time of the request (RFC 1123).
"request"
The request line exactly as it came from the client (RFC 1945).
status
The HTTP status code returned to the client (RFC 1945).
bytes
The content-length of the document transferred.

Internal server errors are recorde in the error log file. The format of this file is a more ad hoc format than the logs using Common Logfile Format, but conforms to the following syntax:

[date] access to path failed for remotehost, reason: reason

4.9 Server Side Includes

Server Side Includes enables the server to run code embedded in HTML pages to generate the response to the client.

Note!

Having the server parse HTML pages is a double edged sword! It can be costly for a heavily loaded server to perform parsing of HTML pages while sending them. Furthermore, it can be considered a security risk to have average users executing commands in the name of the Erlang node user. Carefully consider these items before activating server-side includes.

4.9.1 SERVER-SIDE INCLUDES (SSI) SETUP

The server must be told which filename extensions to be used for the parsed files. These files, while very similar to HTML, are not HTML and are thus not treated the same. Internally, the server uses the magic MIME type text/x-server-parsed-html to identify parsed documents. It will then perform a format conversion to change these files into HTML for the client. Update the mime.types file, as described in the Mime Type Settings, to tell the server which extension to use for parsed files, for example: 

        text/x-server-parsed-html shtml shtm

This makes files ending with .shtml and .shtm into parsed files. Alternatively, if the performance hit is not a problem, all HTML pages can be marked as parsed: 

        text/x-server-parsed-html html htm

4.9.2 Server-Side Includes (SSI) Format

All server-side include directives to the server are formatted as SGML comments within the HTML page. This is in case the document should ever find itself in the client's hands unparsed. Each directive has the following format: 

        <!--#command tag1="value1" tag2="value2" -->

Each command takes different arguments, most only accept one tag at a time. Here is a breakdown of the commands and their associated tags:

config
The config directive controls various aspects of the file parsing. There are two valid tags:
errmsg
controls the message sent back to the client if an error occurred while parsing the document. All errors are logged in the server's error log.
sizefmt
determines the format used to display the size of a file. Valid choices are bytes or abbrev. bytes for a formatted byte count or abbrev for an abbreviated version displaying the number of kilobytes.
include
will insert the text of a document into the parsed document. This command accepts two tags:
virtual
gives a virtual path to a document on the server. Only normal files and other parsed documents can be accessed in this way.
file
gives a pathname relative to the current directory. ../ cannot be used in this pathname, nor can absolute paths. As above, you can send other parsed documents, but you cannot send CGI scripts.
echo
prints the value of one of the include variables (defined below). The only valid tag to this command is var, whose value is the name of the variable you wish to echo.
fsize
prints the size of the specified file. Valid tags are the same as with the include command. The resulting format of this command is subject to the sizefmt parameter to the config command.
flastmod
prints the last modification date of the specified file. Valid tags are the same as with the include command.
exec
executes a given shell command or CGI script. Valid tags are:
cmd
executes the given string using /bin/sh. All of the variables defined below are defined, and can be used in the command.
cgi
executes the given virtual path to a CGI script and includes its output. The server does not perform error checking on the script output.

4.9.3 Server-Side Includes (SSI) Environment Variables

A number of variables are made available to parsed documents. In addition to the CGI variable set, the following variables are made available:

DOCUMENT_NAME
The current filename.
DOCUMENT_URI
The virtual path to this document (such as /docs/tutorials/foo.shtml).
QUERY_STRING_UNESCAPED
The unescaped version of any search query the client sent, with all shell-special characters escaped with \.
DATE_LOCAL
The current date, local time zone.
DATE_GMT
Same as DATE_LOCAL but in Greenwich mean time.
LAST_MODIFIED
The last modification date of the current document.

4.10 The Erlang Webserver API

The process of handling a HTTP request involves several steps such as:

To provide customization and extensibility of the HTTP servers request handling most of these steps are handled by one or more modules that may be replaced or removed at runtime, and ofcourse new ones can be added. For each request all modules will be traversed in the order specified by the modules directive in the server configuration file. Some parts mainly the communication related steps are considered server core functionallity and are not implemented using the Erlang Webserver API. A description of functionality implemented by the Erlang Webserver API is described in the section Inets Webserver Modules.

A module can use data generated by previous modules in the Erlang Webserver API module sequence or generate data to be used by consecutive Erlang Webserver API modules. This is made possible due to an internal list of key-value tuples, also refered to as interaction data.

Note!

Interaction data enforces module dependencies and should be avoided if possible. This means the order of modules in the Modules config directive is significant.

4.10.1 API Description

Each module implements server functionality using the Erlang Webserver API should implement the following call back functions:

The latter functions are needed only when new config directives are to be introduced. For details see httpd(3)

4.11 Inets Webserver Modules

The convention is that all modules implementing some webserver functionallity has the name mod_*. When configuring the webserver an appropriate selection of these modules should be present in the Module directve. Please note that there are some interaction dependencies to take into account so the order of the modules can not be totally random.

4.11.1 mod_action - Filetype/Method-Based Script Execution.

Runs CGI scripts whenever a file of a certain type or HTTP method (See RFC 1945) is requested.

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{new_request_uri, RequestURI}
An alternative RequestURI has been generated.

4.11.2 mod_alias - URL Aliasing

This module makes it possible to map different parts of the host file system into the document tree e.i. creates aliases and redirections.

Exports the following Erlang Webserver API interaction data, if possible:

{real_name, PathData}
PathData is the argument used for API function mod_alias:path/3.

4.11.3 mod_auth - User Authentication

This module provides for basic user authentication using textual files, dets databases as well as mnesia databases.

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{remote_user, User}
The user name with which the user has authenticated himself.

4.11.4 mod_cgi - CGI Scripts

This module handles invoking of CGI scripts

4.11.5 mod_dir - Directories

This module generates an HTML directory listing (Apache-style) if a client sends a request for a directory instead of a file. This module needs to be removed from the Modules config directive if directory listings is unwanted.

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{mime_type, MimeType}
The file suffix of the incoming URL mapped into a MimeType.

4.11.6 mod_disk_log - Logging Using disk_log.

Standard logging using the "Common Logfile Format" and disk_log(3).

Uses the following Erlang Webserver API interaction data, if available:

4.11.7 mod_esi - Erlang Server Interface

This module implements the Erlang Server Interface (ESI) that provides a tight and efficient interface to the execution of Erlang functions.

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{mime_type, MimeType}
The file suffix of the incoming URL mapped into a MimeType

4.11.8 mod_get - Regular GET Requests

This module is responsible for handling GET requests to regular files. GET requests for parts of files is handled by mod_range.

Uses the following Erlang Webserver API interaction data, if available:

4.11.9 mod_head - Regular HEAD Requests

This module is responsible for handling HEAD requests to regular files. HEAD requests for dynamic content is handled by each module responsible for dynamic content.

Uses the following Erlang Webserver API interaction data, if available:

4.11.10 mod_htacess - User Configurable Access

This module provides per-directory user configurable access control.

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{remote_user_name, User}
The user name with which the user has authenticated himself.

4.11.11 mod_include - SSI

This module makes it possible to expand "macros" embedded in HTML pages before they are delivered to the client, that is Server-Side Includes (SSI).

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{mime_type, MimeType}
The file suffix of the incoming URL mapped into a MimeType as defined in the Mime Type Settings section.

4.11.12 mod_log - Logging Using Text Files.

Standard logging using the "Common Logfile Format" and text files.

Uses the following Erlang Webserver API interaction data, if available:

4.11.13 mod_range - Requests with Range Headers

This module response to requests for one or many ranges of a file. This is especially useful when downloading large files, since a broken download may be resumed.

Note that request for multiple parts of a document will report a size of zero to the log file.

Uses the following Erlang Webserver API interaction data, if available:

4.11.14 mod_response_control - Requests with If* Headers

This module controls that the conditions in the requests is fullfilled. For example a request may specify that the answer only is of interest if the content is unchanged since last retrieval. Or if the content is changed the range-request shall be converted to a request for the whole file instead.

If a client sends more then one of the header fields that restricts the servers right to respond, the standard does not specify how this shall be handled. httpd will control each field in the following order and if one of the fields not match the current state the request will be rejected with a proper response.
1.If-modified
2.If-Unmodified
3.If-Match
4.If-Nomatch

Uses the following Erlang Webserver API interaction data, if available:

Exports the following Erlang Webserver API interaction data, if possible:

{if_range, send_file}
The conditions for the range request was not fullfilled. The response must not be treated as a range request, instead it must be treated as a ordinary get request.

4.11.15 mod_security - Security Filter

This module serves as a filter for authenticated requests handled in mod_auth. It provides possibility to restrict users from access for a specified amount of time if they fail to authenticate several times. It logs failed authentication as well as blocking of users, and it also calls a configurable call-back module when the events occur.

There is also an API to manually block, unblock and list blocked users or users, who have been authenticated within a configurable amount of time.

4.11.16 mod_trace - TRACE Request

mod_trace is responsible for handling of TRACE requests. Trace is a new request method in HTTP/1.1. The intended use of trace requests is for testing. The body of the trace response is the request message that the responding Web server or proxy received.

Copyright © 1991-2007 Ericsson AB