Tutorial 03 :
Basic HTTP Sever

Proxy Server Download Section

We will see how HTTP server/proxy works and how to implement basic servers.
May be FTP will be added in the future ...

This tutorial is based on Tutorial 01.
 

A proxy takes place between a client and the server.
The proxy is the server for the client, he is the client for the server.


Proxy

Proxy can be use to limite access to the net (prohibit some site ...), anonymous surfing (IP is the proxy IP not yours) ...

The majority of anonymous internet surfing applications are based on using proxies for hiding the IP of the client. Destination server should see the IP of the proxy.
 

The proxy is between the client and the server, so all data flow pass throw the proxy. So, we can catch queries of the client and the response of the server.

So, if you set up your internet browser to use your proxy, you can catch the queries of your navigator with its associted response (will be used bellow).
This can be done for FTP and other kind of connections.
 

First a proxy have a server side, effectively he's the server for the client.
A ServerSocket is listening on a defined port, for client connection. HTTP proxies usually listen on port 8080.
 

Server side (1)

    //Connecting server
    proxy = new ServerConnection<ListenerServerLink>(PORT);
    ...    //See serverside (2)
    if(!proxy.connect())
    {
        //Connection failed : occurs if the port is in use ...
        System.exit(-1);
    }

    //Listen for client connection
    int index = -1;
    while((index = proxy.acceptConnection()) != -1)
    {
        //When a client is connected, start query listener thread
        proxy.getLink(index).getListener().start();
    }

Processing of the query of the client is done in the listener loop. Just after the connection is done, the thread is starten and the server continue listening for another client request.
Using thread allow multiple connection of clients at a time.

When a client is connected (acceptConnection above), a ListenerServerLink object is created storing the connection Socket.
We add to this object a listener thread in which we wait for the request of the client. Then, when the datas are received, we process inputs usinf the proxy protocol (ie process what should be done with datas).
 

Server side (2)

proxy.setServerLinkCreator(new ServerLinkCreator<ListenerServerLink>(){
    public ListenerServerLink create(Socket socket) throws IOException
    {
        ListenerServerLink client = new ListenerServerLink(socket);
        //Attach listener thread
        client.setListener(new Thread() {
            public void run()
            {
                /*
                 * Listen for a query received,
                 * call process input when some datas are received
                 */
                ...
            });
        //Attach the Proxy protocol
        client.setProtocol(new Protocol() {
            public void processInput(InputStream in)
            {
                //Process query
                ...
            });
        return client;
    }
});

Here, we have received the query of the client. The proxy should transmit it to the server :

The destination server is usually specified in the client's query. So, todetermine it, we have to know which kind of query (HTTP/1.1, FTP ...) depending on the proxy kind.
For an HTTP proxy protocol, the first line of the query is :
   Query http://host:port/path/file.ext HTTP/1.1
We just have to take the second token an extract host:port from itand we have the destination !

Here is the simplified code of the proxy protocol, to view full code read the source of this tutorial.
 

Proxy Protocol

    //Read query
    Vector<byte[]> vector = new Vector<byte[]>();
    while(in.available() > 0)
    {
        byte[] bytes = client.readInput();    //See Tutorial 01
        vector.add(bytes);
    }

    //Parse query for host/port of the server
    host = ...;
    port = ...;

    //Connect to the server
    ClientConnection server = new ClientConnection(InetAddress.getByName(host), port);
    server.connect()

    //Transmit query received
    for(byte[] bytes : vector)
        server.send(bytes);    //See Tutorial 01

    //Transmit response of the server
    while(server.hasInput() > 0)
        client.send(server.readInput());
   
    //Disconnection
    server.disconnect();
    proxy.disconnectLink(client);

 

Run the http proxy on the server machine :

HTTP Proxy start-up

//Bind the proxy on the port 888
 java -classpath Tutorial03-Server-Proxy.jar org.jouvieje.network.proxy.http.HTTPProxy 8888

/*
 * Redirection feature, all requested ressources on this ip will be considered by the proxy as localhost.
 * This feature is usefull in case of the proxy is running on the same machine with a local http server (server not accessible from outside).
 * I use this proxy typically to access my local apache/php server from another computer (Linux/MacOsX).
 */
 java -classpath Tutorial03-Server-Proxy.jar org.jouvieje.network.proxy.http.HTTPProxy 8888 169.254.192.146

In the client machine, set-up your web browser by entering the proxy server IP and the proxy port.
The proxy server is the ip of the server machine (127.0.0.1, server and client are the same machine here), the port is the port passed above (8888 here).

Here is a (french) screenshot of the connection settings of firefox :


Proxy settings under Firefox


 

Before writting an http server, you should know a little on HTTP 1.1 protocol (see RFC 2616 for more informations).
The objectif of this tutorial is not to write a full HTTP server, we will only see basis of an http server (most used features).
 

This query is used to retrieve ressources from the server.
It is basically used for asking some web page, pictures, dowload files ...

Form of the query :
 GET ressource HTTP/1.1
 ...   //some options in relation to the query

Here is a capture of the traffic generated by your browser for requesting this page.
This capture was taken from the HTTP proxy from the first part of this tutorial.
 

Query

GET /Java/Network/Tutorials/Tutorial3.php HTTP/1.1
Host: jerome.jouvie.free.fr
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive
Cache-Control: max-age=0
 

 

Reply

HTTP/1.1 200 OK
Date: Mon, 17 Jul 2006 15:45:13 GMT
Server: Apache/1.3.33 (Win32) PHP/4.3.10
X-Powered-By: PHP/4.3.10
Transfer-Encoding: chunked
Content-Type: text/html

3FE1
<HTML>
<HEAD>
...
</BODY>
</HTML>
0

 

The first line contains the reply code, here it is 200 OK (no error). There are other codes like file not found (404 Not Found) ...

Following lines indicates the date of reply. It also indicates that the server is an Apache powered by PHP.

Next two lines defines the transfert type and the MIME format of the ressource. With the MIME format, the navigator can determinate which application to use when receiving the file (ex: mime type of jnlp file is application/x-java-jnlp-file associated to Java Web Start).

This is followed by a blank line, and the number of bytes (only for chunked transfert type) of the ressources in hexadecimal base (base 16).
Afterwards, the ressource is sended and followed by 0 (only for chunked transfert type).
 

The ressource asked can be a folder (ex: GET / HTTP/1.1).
Datas to send is either the index.html file in this folder or, if this file don't exists, an html file generated listing files and directories.

Here is an exemple of an html page generated by the server listing files and dirs :

Listing file & directories

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
<HEAD>
<TITLE>Index of /</TITLE>
</HEAD>
<BODY>
<H1>Index of /</H1>
<PRE><IMG SRC="" ALT=" "> <A HREF="?N=D">Name</A> <A HREF="?M=A">Last modified</A> <A HREF="?S=A">Size</A> <A HREF="?D=A">Description</A>

<HR>
<img src="" alt="[DIR]"> <a href="images/">images/</a> 18/11/05 10:26 -
<img src="" alt="[ ]"> <a href="index.html">page.html</a> 24/08/05 15:47 40k
</PRE><HR>
<ADDRESS>Jouvieje's Basic HTTP Server at <A HREF="http://127.0.0.1:2222">127.0.0.1</A> Port 2222</ADDRESS>
</BODY></HTML>

 

Here is a capture for larger files like image, archives ...

GET /images/Fond.png HTTP/1.1

HTTP/1.1 200 OK
Date: Mon, 17 Jul 2006 16:44:08 GMT
Server: Apache/1.3.33 (Win32) PHP/4.3.10
Last-Modified: Tue, 13 Jul 2004 13:47:58 GMT
ETag: "0-e4-40f3e80e"
Accept-Ranges: bytes
Content-Length: 228
Content-Type: image/png

...   //the ressource

Content length indicates the number of bytes of the file.
Accept ranges allow partial transfert of a ressource. ETag is a tag associated to the ressource. The navigator can only ask a piece of a file, usefull for incomplete transfert (see 206 Partial Content).

Here is an example of partial transfert :
 

206 Partial Content

//Query
GET ... HTTP/1.1
...
Range: bytes=572813-
If-Range: "0-1cd0cb-443a9d3a"

//Reply
HTTP/1.1 206 Partial Content
Date: Fri, 14 Jul 2006 13:19:28 GMT
Server: Apache/1.3.33 (Win32) PHP/4.3.10
Last-Modified: Mon, 10 Apr 2006 18:00:26 GMT
ETag: "0-1cd0cb-443a9d3a"
Accept-Ranges: bytes
Content-Length: 1315646
Content-Range: bytes 572813-1888458/1888459
Content-Type: application/octet-stream    //For exe files

...    //the 572813th byte to the end (first byte is 0)

 

This query used to send some datas to a ressource. This is commonly used by form in html page.

This following form ask for the pseudo of the visitor and send it to the page form.php

Form (example)

<html>
 <body>
  <form action="form.php" method="post">
   Your pseudo: <input name="pseudo" size="25" type="text">
   <input value="Send" type="submit">
  </form>
 </body>
</html>

The query generated by the navigator look like :

Query

POST /Form/form.php HTTP/1.1
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive
Referer: http://127.0.0.1/Form/Index.html
Content-Type: application/x-www-form-urlencoded
Content-Length: 15

Pseudo=mypseudo

Content type is application/x-www-form-urlencoded, length is 15 bytes.
The server should send "Pseudo=mypseudo" to the page form.php, then send the result to the client. This is commonly used associated with php, cgi ...

Here, we write a basic server with no extensions like php ... We will send error code 405 Method Not Allowed.

405 Method Not Allowed

HTTP/1.1 405 Method Not Allowed
Date: Sun, 16 Jul 2006 14:38:33 GMT
Server: Apache/1.3.33 (Win32) PHP/4.3.10
Allow: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, PATCH, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK, TRACE
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1

150
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>405 Method Not Allowed</TITLE>
</HEAD><BODY>
<H1>Method Not Allowed</H1>
The requested method POST is not allowed for the URL /Form/Index.html.<P>
<HR>
<ADDRESS>Apache/1.3.33 Server at <A HREF="mailto:admin@localhost">127.0.0.1</A> Port 80</ADDRESS>
</BODY></HTML>

0

 

Here is the synthesis of the implemented part of HTTP 1.1 for our server.
 

HTTP Protocol

//Read input datas
String ins = "";
while(in.available() > 0)
    ins += new String(client.readInput());
//LineTokenizer
Vector<String> request = new Vector<String>();
int index = -1;
int from = 0;
while((index = ins.indexOf(LINE_SEPARATOR, from)) > -1)
{
    request.add(ins.substring(from, index));
    from = index+1;
}
request.add(ins.substring(from));

//Read query and ressource
StringTokenizer tokens = new StringTokenizer(request.get(0));
String query = tokens.nextToken();
String ressource = tokens.nextToken();

if(query.equals("GET"))
{
    File file = ...;
    if(file.exists())
    {
        if(file.isFile())
        {
            if(Ask a range)
            {
                //Send 206 Partial Content
                ...
            }
            else
            {
                //Send 200 OK
                ...
            }
        }
        else
        {
            //Generate list of files/directories
            ...
           
            //Send 200 OK
            ...
        }
    }
    else
    {
        //Send 404 Not Found
        ...
    }
}
if(query.equals("POST"))
{
    //Send 405 Method Not Allowed
    ...
}
else
{
    //To be implemented !
}
 

 

 

Previous Tutorial

Back

Next turorial

Last modified on 01/07/2010
Copyright © 2004-2010 Jérôme JOUVIE - All rights reserved. http://jerome.jouvie.free.fr/