diff --git a/wiki/Manual2.wiki b/wiki/Manual2.wiki index 606ef34..8ff6fcc 100644 --- a/wiki/Manual2.wiki +++ b/wiki/Manual2.wiki @@ -12,6 +12,106 @@ To use PPSS in a distributed fasion, The following steps must be performed: # Deploy PPSS to the nodes. # Start PPSS on all nodes. += A list of all configuration options = + +{{{ +bash-3.2$ ./ppss.sh --help + +|P|P|S|S| Distributed Parallel Processing Shell Script 2.40 + +PPSS is a Bash shell script that executes commands in parallel on a set +of items, such as files in a directory, or lines in a file. + +Usage: ./ppss.sh [ MODE ] [ options ] + +Modes are optional and mainly used for running in distributed mode. Modes are: + + config Generate a config file based on the supplied option parameters. + deploy Deploy PPSS and related files on the specified nodes. + erase Erase PPSS and related files from the specified nodes. + + start Starting PPSS on nodes. + pause Pausing PPSS on all nodes. + stop Stopping PPSS on all nodes. + node Running PPSS as a node, requires additional options. + +Options are: + +--command | -c Command to execute. Syntax: ' ' including the single quotes. + Example: -c 'ls -alh '. It is also possible to specify where an item + must be inserted: 'cp "$ITEM" /somedir'. + +--sourcedir | -d Directory that contains files that must be processed. Individual files + are fed as an argument to the command that has been specified with -c. + +--sourcefile | -f Each single line of the supplied file will be fed as an item to the + command that has been specified with -c. + +--config | -C If the mode is config, a config file with the specified name will be + generated based on all the options specified. In the other modes. + this option will result in PPSS reading the config file and start + processing items based on the settings of this file. + +--enable-ht | -j Enable hyperthreading. Is disabled by default. + +--log | -l Sets the name of the log file. The default is ppss-log.txt. + +--processes | -p Start the specified number of processes. Ignore the number of available + CPUs. + +--delay | -D Adds an initial random delay to the start of all parallel jobs to spread + the load. The delay is only used at the start of all 'threads'. + +The following options are used for distributed execution of PPSS. + +--master | -m Specifies the SSH server that is used for communication between nodes. + Using SSH, file locks are created, informing other nodes that an item + is locked. Also, often items, such as files, reside on this host. SCP + is used to transfer files from this host to nodes for local procesing. + +--node | -n File containig a list of nodes that act as PPSS clients. One IP / DNS + name per line. + +--key | -k The SSH key that a node uses to connect to the master. + +--known-hosts | -K The file that contains the server public key. Can often be found on + hosts that already once connected to the server. See the file + ~/.ssh/known_hosts or else, manualy connect once and check this file. + +--user | -u The SSH user name that is used when logging in into the master SSH + server. + +--script | -S Specifies the script/program that must be copied to the nodes for + execution through PPSS. Only used in the deploy mode. + This option should be specified if necessary when generating a config. + +--transfer | -t This option specifies that an item will be downloaded by the node + from the server or share to the local node for processing. + +--no-scp | -b Do not use scp for downloading items. Use cp instead. Assumes that a + network file system (NFS/SMB) is mounted under a local mountpoint. + +--outputdir | -o Directory on server where processed files are put. If the result of + encoding a wav file is an mp3 file, the mp3 file is put in the + directory specified with this option. + +--homedir | -H Directory in which directory PPSS is installed on the node. + Default is 'ppss'. + +Example: encoding some wav files to mp3 using lame: + +./ppss.sh -c 'lame ' -d /path/to/wavfiles -j + +Running PPSS based on a configuration file. + +./ppss.sh -C config.cfg + +Running PPSS on a client as part of a cluster. + +./ppss.sh -d /somedir -c 'cp /some/destination' -s 10.0.0.50 -u ppss -t -k ppss-key.key + +}}} + = Preparation of server and nodes = The following preparations must be made in order to use PPSS in a distributed fasion: