Noaaport/Nbsp DokuWiki

See Note

This file describes the alternatives to the default disk-based spool directory that holds the raw data files that are received and later sent to the filter and network server for processing and distribution.

Nbsp supports the following spool configurations

• File system spool
• Private memory based bdb spool

Some configurations are more suitable than others for special purposes. For example, the Memory based spool is suitable for an ingest machine that will not do any processing itself, but will act as a master to one or more processing slaves. The default mode is the file system spool. The other one is explained below.

In a master-slave configuration involving two or more machines, the master can be configured such that the spool directory resides entirely in memory. The motivation behind this is to have the master machine dedicated to only receive the noaaport data and transfer it, by tcp, to the slaves, which will then execute the filters to build the data products. In this way the master machine is freed from the demands of the Nbsp filters and associated i/o disk requirements, which can be more efficient in minimizing the loss of the multicast packets.

In this mode of operation, the master can only serve NBS1 type clients (not NBS2 nor emwin clients). In addition, the processing filters must disabled, because the spool resides in a memory sector that is private to the server and it is not accessible by the filters.

The recommended way to enable the memory bdb spool is:

1) Set the variable (see note below for the difference)

	set feature(spooltype) 2

or

	set feature(spooltype) 3

in

• features.conf

2) Copy the files dist/nbspd-mspoolbdb.conf-ex to site/nbspd.conf and dist/netfilter-mspoolbdb.conf-ex to site/netfilter.conf, and follow the instructions there.

3) Note that any additional configuration settings that are needed in individual setups must still be done either in the main nbspd.conf file or in the site/nbspd.conf (for example, setting the interfaces, port numbers, and so on).

This method ensures a safe starting point to configure the master with the memory spool. The only filters that can be enabled are the inventory filter, and the netfilter which can be used by the network server to decide what products are sent to what slaves. The above steps enable these two filters properly, but the relevant lines in the file nbspd-mspoolbdb.conf (renamed site/nbspd.conf) can be commented to disable them as well if desired.

The difference between the settings 2 and 3 is that in the first case the bsd resides entire in memory. It is a truly memory based bdb. In the second case (setting 3), the db is in memory but the bdb administrative bdb enviroment files are reside in the file system. This second setting could result in slightly slower performance, but it is less prone to the db_recovery type of errors to which the pure in-memory bdb is susceptible.