Go to the first, previous, next, last section, table of contents.


Introduction to STORE

As described in the brief introduction, STORE is mostly a set of concepts and conventions on how to organize and install program packages. The visible parts of STORE itself consists of:

  1. A directory tree structure.
  2. A set of utility programs.

We will start by describing the directory tree structure.

The /store directory tree

The `/store' directory tree looks much like a typical `/local' or `/usr' directory tree on a UNIX system (i.e. there is a `/store/bin' directory, a `/store/lib' directory, and so on).

                                  /store
                                     |
  +----+----+----+-- ... --+---------+-------------------+
  |    |    |    |         |                             |
 bin  lib  man  etc       doc                          store
            |                                    +-------+-------------+
           man1                                  |                     |
                                                khym                  Adm
                               +-----+-----+-----+-- ... --+
                               |     |     |     |         |
                              fbm  emacs  gcc   tex       zoo

No files reside in the immediate subdirectories of `/store'. Instead we find cryptic soft links, like (if we are on a Sun Sparc machine):

 /store/bin/gcc -> /store/store/khym/gcc/ver-2.3.3/bin/gcc@sun4os4

Alternatively, if we are on a DecStation running Ultrix 4.x (1):

 /store/bin/gcc -> /store/store/khym/gcc/ver-2.3.3/bin/gcc@dsult4

The observant reader may have noticed the system in the files pointed to by the two softlinks, and (correctly) concluded that the directory tree under `/store/store/khym/gcc' may contain some interesting material. So let us take a look at this directory tree:

                         /store
                            |
                          store/
                            |
            ...--+----------+---------+---------+---...
                 |          |         |         |
               lise1/      khym/     ild/     storlind/
                            |
                           gcc/
                            |
     +------------+---------+-+-----------+-------------------+
     |            |           |           |                   |
 registration ver-2.3.3/ src-2.3.3/ src-2.3.3-sun4os4/ src-2.3.3-dsult4/
                  |
                  |
         +--------+---+------------+
         |            |            |
        bin/         lib/        emacs/
         |                         |
     gcc@sun4os4                info/
     gcc@dsult4                   |
                                gcc.info
                                ...

What this directory shows us is that no actual files "live" in the `/store' sub-directories. Instead we have soft links pointing to files collected in applications. The name of the `/store' tree used in the STORE documentation is linktree or server linktree, and conceptually, the macro view. The directory tree where the files actually reside is called a package's version tree or the micro view.

The scheme gives us some advantages: First, we don't get any leftover files on deinstallation(2), always a problem when upgrading packages.

Another advantage is that we can have several versions of a package compiled at any given time and install/deinstall different versions with a few simple commands. We can also have softlinks pointing to packages on remote systems, saving space on our local system (but decreasing performance and reliability). By looking at the softlinks, it is immediately apparent to an experienced STORE user what application files belongs to, what version of the application, and where to go to find all the files belonging to the application. Even better, this procudure may be automated, so less sophisticated users may find their way around. (NOTE: I have planned to write this automated script, but I haven't gotten around to it yet).

An application (or package) is a directory containing:

In the level above khym in the figure, the other STORE file servers are mounted on the machine khym (with khym appearing on the same level in their STORE directory trees).

STORE also supports shadowing of applications. If we take a look at a typical application (in this case gcc, the GNU C compiler):

                         /store
                            |
                          store/
                            |
...--+----------+---------+-+--------------------------+---...
     |          |         |                            |
   lise1/      khym/     ild/                      storlind/
                |                                      |
               gcc/                                  .gcc/
                |                                      |
     +----------+-+------------+-...            +------+-----+
     |            |            |                |            |
 registration ver-2.3.3/  src-2.3.3/       registration ver-2.3.3/
                  |                                          |
                  |                                          |
         +--------+-...                             +--------+-...
         |                                          |
        bin/                                       bin/
         |                                          |
     gcc@sun4os4                               gcc@sun4os4
     gcc@sun3os4                               gcc@sun3os4
     gcc@dsult4                                ...
     ...

The file server khym holds the master installation of the application gcc. Any changes to the current version, or installation of new versions of the package should take place here.

The file server storlind holds a slave of the gcc package. The slave copy consists of the `registration' file and a copy of the `ver-<version>' directory tree holding the installed files of the package.

Because khym holds the master installation it needs to hold files for all architectures using this particular system. In this example, storlind is a file server for a site using Sun 4 and Sun 3 machines, which means that only files for these arhictectures, as well as those common for all architectures (e.g. emacs info files) are copied over.

The updating of slave installation of packages is done nightly. The reason for using slave packages is to reduce cross-system NFS access (a STORE system may use a wide area network).

The main utility programs.

The main utility programs are those that do batch processing or the STORE trees, checking consistency, propagating changes, and updating linktrees. The basic functionality of these has been unchanged since the first versions. See section Understanding STORE output.

cmaster utility

cmaster runs on each machine having a repository. It checks all the locally defined repositories and finds master applications. It reads their registration files, checks the application for consistency (potentially producing warnings that need to be dealt with), and writes back the registration file. Any output from cmaster probably indicates a real problem with the application.

cslave utility

cslave finds all slave applications on the local STORE server, and fetches changes or new versions from the master servers. Since the master copies of applications will change with time (if the master copy is properly maintained and bugs are fixed!), some output is inevitable. Also, spurious errors may indicate NFS problems or incorrect mounting of filesystems. It is hard at first to recognize which warnings from cslave need manual intervention, but running it manually a couple of times should give some clues. Since it fixes most of the transient problems on the first run, any warnings or errors on the second run may indicate a "permanent" problem, especially if you also make sure that all the repositories involved have been checked with cmaster first.

cclient utility

cclient first processes the linktree to find links that are obviously wrong (like pointing at nonexistant files), or extra non-symlinks. Then, it finds all available applications for the linktree and makes the right symlinks in the linktree, optionally running any "nightly commands" defined by the applications.

report utility

The report utility finds all applications and makes a nice report on them, including a graphic presentation of architectures supported and special files. Most of the information reported is extracted from the registration file directly.

status utility (planned)

The status utility is even more administrator-oriented than the report utility. It will show the amount of disk space occupied by different applications, what types of documentation is available, what source code and build trees are left around, etc. It is intended to help prevent "bit rot" in the STORE system.

autocomp utility

Autocomp is a tool designed to make things easier for people updating applications. It will compile and install applications for different architectures automatically(and simultaneously). Of course, if compiling or installing errors occur, autocomp will fail. (A mail will be sent to notify the one who started it.) Autocomp tries to pick the least loaded host for compilation, and makes sure only one autocomp process is installing in a linktree at a time. It depends on rsh/rcp access from the host autocomp is started to all the hosts that will be used for the actual compilation.


Go to the first, previous, next, last section, table of contents.