\chapter{Resources}

In the previous chapter we have learned how to work with URIs, objects that
identify resources; but how to manipulate the resources themselves?

The problem we face now is the fact that the resources are dispersed, stored
in different systems, and accessed through different protocols. In spite of
that, it would be nice to be able to manipulate them with a uniform API.

That's the purpose of {\tt itools.resources}, to provide an abstraction layer
over resources: doesn't matters where the resources are stored (in the local
file system, in a remote web server, etc.), they are handled with the same
consistent API.

\section{Retrieving resources}

Of course, the first step, before working with a resource, is to retrieve it,
this is done with the function {\tt get\_resource}:

\begin{api}
  {\tt get\_resource(reference)}\\
  - From the given URI reference returns a resource object.
\end{api}

Let's see a few examples:

\begin{code}
    >>> from itools.resources import get_resource
    >>> get_resource('examples/hello.txt')
    <itools.resources.file.File instance at 0x402175ac>
    >>> get_resource('http://example.com')
    <itools.resources.http.File instance at 0x404aadec>
    >>> get_resource('examples')
    <itools.resources.file.Folder instance at 0x401e568c>
\end{code}

These examples show two fundamental aspects of {\tt itools.resources} which
we will explore on detail later. First is the support for multiple protocols,
in particular the example illustrates the schemes {\tt file} for the local
file system, and {\tt http} for the {\bf H}yper{\bf T}ext {\bf T}ransfer
{\bf P}rotocol.

The second aspect is the support for two kinds of resources, files and folders.
Before looking at each type of resource, files and folders, let's see the API
they share:

\begin{api}
  {\tt uri}\\
  - The URI reference the resource was retrieved from.

  {\tt get\_mimetype()}\\
  - Tries to guess the mimetype of the resource and returns it as a string.
    Folders are always {\tt application/x-not-regular-file}. Returns {\tt None}
    if it fails.

  {\tt get\_ctime()}\\
  - Returns a {\tt datetime} object with the time the resource was created.

  {\tt get\_mtime()}\\
  - Returns a {\tt datetime} object with the last time the resource was
    modified.

  {\tt get\_atime()}\\
  - Returns a {\tt datetime} object with the last time the resource was
    accessed.
\end{api}

It may happen that a specific scheme does not implements one or more of
these methods.

\section{File Resources}

Besides the common methods for files and folders, each resource type has
a specific API, the one for file resources is:

\begin{api}
  {\tt get\_data()}\\
  - Returns the resource data as a byte string.

  {\tt set\_data(data)}\\
  - Replaces the resource content by the given data (a byte string).

  {\tt get\_size()}\\
  - Returns the length (the number of bytes) of the resource data.

  {\tt \_\_getitem\_\_(index)}\\
  - Returns the byte at the given position.

  {\tt \_\_getslice\_\_(a, b)}\\
  - Returns the byte string that goes from {\em a} to {\em b}.

  {\tt \_\_setitem\_\_(index, value)}\\
  - Sets the given value to the given position ({\em index}). Usually
    value will be just a byte, but it may be a slice too.

  {\tt append(data)}\\
  - Appends the given data to the resource.
\end{api}

As an example of the API let's exercise it with a web page:

\begin{code}
    >>> resource = get_resource('http://example.com')
    >>> print resource
    <itools.resources.http.File instance at 0x404aadec>
    >>> print resource.get_mimetype()
    text/html
    >>> print resource.get_size()
    438
    >>> print resource.get_data()
    <HTML>
    <HEAD>
      <TITLE>Example Web Page</TITLE>
    </HEAD> 
    <body>  
    <p>You have reached this web page by typing
    ...
\end{code}

Of course, in this example, we can not modify the resource as we don't have
write access to the web server.

The methods {\tt \_\_getitem\_\_}, {\tt \_\_setitem\_\_}, etc. are specially
useful to work with binary files, as they allow to read and write data at
specific positions within a resource. These methods allow to access the
resource with the same syntax used to access lists in Python:

\begin{code}
    >>> import struct
    >>> n = struct.pack('>I', 47)
    >>> resource[20:24] = n
\end{code}

This brief example shows how to write the number 47 at the position 20 of
a given resource, encoded as an 32 bits unsigned integer.

For a real example you can look at {\tt itools.catalog}, which makes
extensive and systematic use of this approach.


\section{Folder Resources}

The specific API for folders is:

\begin{api}
  {\tt get\_resource(path)}\\
  - Returns the resource in the given path (where path is either an instance
    of {\tt uri.Path} or a string).

  {\tt get\_resource\_names(path='.')}\\
  - Returns a list with the names of all the resources in the given path.

  {\tt get\_resources(path='.')}\\
  - Returns the resources in the given path (it is a generator).

  {\tt has\_resource(path)}\\
  - Returns {\tt True} if there is a resource in the given path, {\tt False}
    otherwise.

  {\tt set\_resource(path, resource)}\\
  - Adds the given resource to the given path.

  {\tt del\_resource(path)}\\
  - Removes the resource at the given path.

  {\tt del\_resources(paths)}\\
  - Removes the resources at the given paths (where paths is a list of paths).

  {\tt traverse()}\\
  - This method allows to traverse the resource tree below this folder. It
    is a generator which returns a resource at a time, starting by this
    folder.
\end{api}

Let's exercise the API a little:

\begin{code}
    >>> examples = get_resource('examples')
    >>> print examples.get_resource_names()
    ['hello.xhtml~', 'hello.xhtml', 'hello.txt', '.arch-ids']
    >>> hello = examples.get_resource('hello.txt')
    >>> print hello.get_data()
    hello world
\end{code}

To copy a web page to the local file system:

\begin{code}
    >>> tmp = get_resource('/tmp')
    >>> web_page = get_resource('http://example.com')
    >>> tmp.set_resource('example.html', web_page)
    >>> copy_of_web_page = tmp.get_resource('example.html')
    >>> print web_page
    <itools.resources.http.File instance at 0x405e8a2c>
    >>> print copy_of_web_page
    <itools.resources.file.File instance at 0x405fb64c>
\end{code}

To copy a whole tree:

\begin{code}
    >>> from pprint import pprint
    >>> talks = get_resource('/home/jdavid/talks')
    >>> tmp.set_resource('talks', talks)
    >>> pprint(tmp.get_resource_names('talks/EuroPython2004'))
    ['itools-vfs',
     'Makefile',
     'itools-vfs.log',
     'itools-vfs.tex',
     'itools-vfs.aux',
     'itools-vfs.toc',
     'itools-vfs.dvi',
     'itools-vfs.ps',
     'itools-vfs.pdf',
     'itools-vfs.tex~']
\end{code}

This is more impressive when copying a big tree from one storage to another.
For example we use it in the context of the {\bf iKaaro} Content Management
System to export and import web sites from the {\bf Z}ope {\bf O}bject
{\bf D}ata{\bf B}ase to the file system, and back from the file system to the
{\bf ZODB}.


\section{Supported schemes}

At the time of this writing the number of supported schemes is pretty short:
{\tt itools.file} for the file system, {\tt itools.http} for the HTTP
protocol (though the implementation is minimal), and {\tt itools.memory}
to store resources in memory. The {\bf iKaaro} CMS provides support to
store resources in Zope 2.

\subsection{The memory storage}

The memory storage ({\tt itools.memory}) deserves few lines as it is very
handy. It is possible to build a resource from scratch directly calling its
constructor:

\begin{code}
    >>> from itools.resources import memory
    >>> resource = memory.File('hello world')
    >>> resource
    <itools.resources.memory.File instance at 0x405e458c>
    >>> resource.get_data()
    'hello world'
    >>> tmp.set_resource('hello.txt', resource)
\end{code}

This technique is used internally by resource handlers to build handler
instances without passing a resource. They are the object of our next
section.
