Go to the first, previous, next, last section, table of contents.


Concepts and conventions of STORE

When discussing STORE, there are many new terms and concepts. We have tried to define the most important concepts here.

The central repository concept

A repository is a directory containing application packages suitable for STORE. It must be available as /store/store/<name>. You may have several repositories on a single repository server. The naming conventions we use is that repositories are named after the machines.

In the config files, the programs and earlier versions of this documentation the term that was and is used was just "store". This leads to unfortunate confusion between the term when used for the STORE system itself, or when used for the repository concept.

The macro view

The linktree, also called /store, also called the macro view, is what the users see available under the /store directory, not including anything under /store/store.

Machines having one or several repositories

A STORE server is any machine having a repository or a linktree. In most cases, you would have just one repository and one linktree. If you get short on disk space, and add another disk, using it as an extra repository mostly for compiling is easy. You would only want extra linktrees in a heterogenous environment.

The application concept

An application, or package, or software, or a program, is whatever is packaged under a /store/store/<master>/<app> directory. It is wise to keep a single program released as one package as a STORE application. It is generally a very bad idea to have several masters for the same application.

Versions of applications

You may have several versions of an application in the master repository, with different maturity levels. Different version can have different architecture support, so you can compile new versions for just some architectures at first. All slave copies will mimic the master, copying any old or new versions, and deleting versions when they are deleted on the master. Link trees will choose the best version according to their level profile and what support the differing versions have. Keeping old versions around may also provide backup, or be useful references when compiling new versions.

Master repositories

A master repository is a repository containing some masters for applications. In principle, any repository will be checked for masters, but in practice, it is wise to limit the number of master repositories. Master repositories should be on trusted, stable machines, and mounted on most other repository servers to ensure correct copying to slaves.

Slave repositories

A slave repository is a repository with no master versions, only slave copies of applications. You would probably have several slave repositories for each master repository, but as always, this depends on available disk space. Slave copies only hold the version trees for an application, not source code or build trees, and you may limit the amount of files copied by limiting the number of architectures. You only need to make the slave directory /store/store/<slave>/.<app> to make the next nightly run slave down the application, but you may prefer to use slaveapp and linkup.

Compile Servers

A compile server is a machine used for compiling applications in STORE for some architecture. It would probably have a local master or slave repository, but it only needs read-write access to a repository to use for compile trees, and read access to the master servers for the applications it should compile.

Compile trees

A new concept recently introduced in STORE that wasn't originally in the design is the compile tree. This is a bit of a kludge, with some hackery in the tools to support it. But it's very useful.

A master server for an application holds the master registration file, the source code, additional files (doit scripts etc), and probably some shadow trees created with shadow. In the original design, the master should hold shadow trees for all architectures the compilation was compiled for, e.g. sun3, sun4os4, and sgi. The compile server would just mount the repository containing the application read-write, and compile there.

In practice, this isn't feasible. The master server would need to export its repository read-write to a large (and variable) group of compile servers, and compiling on a disk mounted remotely may be nearly impossible. Instead, the compile server creates a pseudo-application for the exact purpose of compiling. If the master is `mast', the compile server `compi', and the application is `someapp' version `1.23', the compile tree would be the pseudo-application `/store/store/compi/someapp-c'.

The -c suffix denotes that this is indeed a compile tree. The shadow utility recognizes this, and looks around in `/store/store/*/someapp' for original source code (in this case, `/store/store/mast/someapp/src-1.23'). The shadow utility makes a shadow tree on the compile server, with links to the NFS-mounted original source. After compilation and successful installation into `/store/store/compi/someapp-c/ver-1.23', the files which differ between the compile server and the master server's version trees should be exactly the architecture-dependant files. These can be tar'ed together and transferred to the master server to complete the installation for this specific architecture. (Postinst will try to do this tar automatically).

Generalized architectures

In STORE, an architecture is a description of a set of computers. It may range from any machine to one very specific machine. The name "architecture" should indicate the common meaning, but the concept is deeper than this. For example, "sparc" and "sgi" is two different architectures, but "sparc" may be further subdivided into "sun4os4" or even into "sun4mos413p", indicating a SparcStation 10 series running SunOS 4.1.3 with patches.

For files, architectures are denoted by suffixes using an @ as a special separator. Mostly, we are only interested in binary files having support for a special hardware/software platform. However, it is sometimes necessary to have configuration files that depend on other attributes of machines, for example mail programs that need to know the local address. The architecture concept, therefore, has been expanded to include other attribute-value pairs. The currently defined attributes are:

architecture
-- arch:<value> or only <value>
domain
-- domain:<value> or d:<value>

So a file for SparcStations running SunOS 4 could be `file@sun4os4', while a file for machines at the mathematics department could be `file@domain:imf.unit.no'. In the pathological case of a file with compiled-in configuration, one would need `file@arch:sun4os4,domain:imf.unit.no'. This general syntax would also permit additional attributes to be defined, but then some of the internals of the STORE scripts would have to be changed, too.

The administrative area

The home area for the "store" user contains configuration files for STORE, logs, reports, and may also have special programs to avoid peculiar troubles on special systems.

It is probably a good idea to share the administrative area between all machines running STORE in an administrative domain. This may be done by making `/store/store/Adm' a symlink to `/store/store/<primary>/Adm', where `<primary>' is the main repository server in the administrative domain. The problem is, of course, that if the primary server is down, the directory is unavailable, and you cannot run the STORE scripts, not even manually. However, it makes for much easier administration, as you otherwise would have to keep configuration files up-to-date on several servers.

Dot files: As all users, the "store" user should have a good setup for the shell in use, such as .cshrc/.login or .bashrc/.bash_profile files which are distributed in the inistore package. The most important thing about these files is setting the PATH variable properly. This must be done individually on different systems, since machines, OSes and sites differ substantially in their proper setup. The distributed set will try to source files in a common area, as this is the most widespread method (and also one of the best) for the setup of the shell. In addition, the distributed files will source files provided as part of the STORE scripts, to make STORE support commands available. These are implemented as perl scripts, and directly available as aliases in the shell. (See `/store/etc/internal/aliases.{csh,bash}' for details).

Configuration files are contained in the `etc/' directory. The main configuration file is just the `config' file. See section Configuration file.

The `initials' is part of the perl-internal application, and is a site-wide file contains initials for the persons installing and maintaining applications in STORE, with mail address, organization, etc. The primary repository maintainers should update this in the same way as the `genarchs' file whenever new staff is hired. See section The genarchs configuration file.

There are also descriptions of the different licensing terms that applies to the programs you put into STORE, and the classifications of programs that you use. These will also be maintained in the same way as the `initials' file.

The `logs/' directory contains logs from the nightly runs of the STORE programs, and may also be used for other jobs that are run via the nightly commands feature.

The `bin/' directory is used mainly for holding the nightly script `night.sh' that runs the STORE updates. This needs to be modified to be able to: Have a correct PATH, start the STORE updates (commonly via rsh or remsh), and send mail about the result. If you need to make script versions of common programs to avoid errors or deficiencies in your operating system, this is also the place to put such scripts, so they will be found by the STORE programs.

Cooperating administrative domains

An administrative domain is a set of repository servers administrated together, with a single configuration file. They would probably be located nearby each other net-wise, so it is possible to use any repository from any linktree inside the administrative domain. One could define the administrative domain as the set of machines running the STORE scripts on the same nightly run.

The set of administrative domains NFS-mounting directories and slaving applications from each other is a STORE site. The site's primary repository is the master for the perl-internal application. You don't need to mount every other repository directly - you can slave applications indirectly.

Different nightly scripts

The STORE nightly job is run on the primary server for an administrative domain. It will start the cmaster and cslave scripts on all repository servers in the domain, and the cclient script on all machines hosting a link tree, by using the `rsh' command. In addition, it will run the report script on the primary server, to make a report on all available programs. Any nightly jobs specified in the registration files will be run (by cclient) for each linktree, with the TOPDIR environment variable set to the top of the linktree.

Using persons' initials for reporting

To make the reports produced by STORE more readable, initials should be registered in the registration files. The standard report script will work well with two- to four-letter acronyms, while longer names will be truncated to four letters.


Go to the first, previous, next, last section, table of contents.