From 6a2fedae412889137b33e35af9ae8a6e6231c271 Mon Sep 17 00:00:00 2001 From: Louwrentius Date: Sun, 8 Nov 2009 18:32:43 +0000 Subject: [PATCH] Edited wiki page through web user interface. --- wiki/Manual1.wiki | 95 ++++++++++++++++------------------------------- 1 file changed, 32 insertions(+), 63 deletions(-) diff --git a/wiki/Manual1.wiki b/wiki/Manual1.wiki index a6818f2..dc89c8d 100644 --- a/wiki/Manual1.wiki +++ b/wiki/Manual1.wiki @@ -11,29 +11,35 @@ The following output is displayed by PPSS when executed without any options: {{{ -'Distributed Parallel Processing Shell Script -Version: 2.0 +bash-3.2$ ./ppss.sh + +|P|P|S|S| Distributed Parallel Processing Shell Script 2.40 + +usage: ./ppss.sh [ -d | -f ] [ -c ' "$ITEM"' ] + [ -C ] [ -j ] [ -l ] [ -p <# jobs> ] + [ -D ] [ -h ] [ --help ] + +Examples: + ./ppss.sh -d /dir/with/some/files -c 'gzip ' + ./ppss.sh -d /dir/with/some/files -c 'gzip "$ITEM"' -D 5 + ./ppss.sh -d /dir/with/some/files -c 'cp "$ITEM" /tmp' -p 2 +}}} + +If PPSS is executed with -h, more options are displayed. + +{{{ +bash-3.2$ ./ppss.sh -h + +|P|P|S|S| - Distributed Parallel Processing Shell Script - +Version: 2.40 PPSS is a Bash shell script that executes commands in parallel on a set -of items, such as files, or lines in a file. +of items, such as files in a directory, or lines in a file. -Usage: ./ppss.sh MODE [ options ] - or -Usage: ./ppss.sh MODE -c +This short summary only discuesses options for stand-alone mode. for all +options, run PPSS with the options --help -Modes are: - - standalone For execution of PPSS on a single host. - node For execution of PPSS on a node, that is part of a 'cluster'. - config Generate a config file based on the supplied option parameters. - deploy Deploy PPSS and related files on the specified nodes. - erase Erase PPSS and related files from the specified nodes. - - start Starting PPSS on nodes. - pause Pausing PPSS on all nodes. - stop Stopping PPSS on all nodes. - -Options are: +Usage ./ppss.sh [ options ] --command | -c Command to execute. Syntax: ' ' including the single quotes. Example: -c 'ls -alh '. It is also possible to specify where an item @@ -50,61 +56,24 @@ Options are: this option will result in PPSS reading the config file and start processing items based on the settings of this file. ---disable-ht | -j Disable hyperthreading. Is enabled by default. +--enable-ht | -j Enable hyperthreading. Is disabled by default. --log | -l Sets the name of the log file. The default is ppss-log.txt. --processes | -p Start the specified number of processes. Ignore the number of available - CPUs. + CPU's. ---force | -F Forces PPSS to start even if another instance is already running. This causes all instances - of PPSS to keep on running. They will process items, but not stop. - -The following options are used for distributed execution of PPSS. - ---server | -s Specifies the SSH server that is used for communication between nodes. - Using SSH, file locks are created, informing other nodes that an item - is locked. Also, often items, such as files, reside on this host. SCP - is used to transfer files from this host to nodes for local procesing. - ---node | -n File containig a list of nodes that act as PPSS clients. One IP / DNS - name per line. - ---key | -k The SSH key that a node uses to connect to the server. - ---known-hosts | -K The file that contains the server public key. Can often be found on - hosts that already once connected to the server. See the file - ~/.ssh/known_hosts or else, manualy connect once and check this file. - ---user | -u The SSH user name that is used when logging in into the master SSH - server. - ---script | -s Specifies the script/program that must be copied to the nodes for - execution through PPSS. Only used in the deploy mode. - This option should be specified if necessary when generating a config. - ---transfer | -t This option specifies that an item will be downloaded by the node - from the server or share to the local node for processing. - ---no-scp | -b Do not use scp for downloading items. Use cp instead. Assumes that a - network file system (NFS/SMB) is mounted under a local mountpoint. - ---outputdir | -o Directory on server where processed files are put. If the result of - encoding a wav file is an mp3 file, the mp3 file is put in the - directory specified with this option. +--delay | -D Adds an initial random delay to the start of all parallel jobs to spread + the load. The delay is only used at the start of all 'threads'. Example: encoding some wav files to mp3 using lame: -./ppss.sh standalone -c 'lame ' -d /path/to/wavfiles -j +./ppss.sh -d /path/to/wavfiles -c 'lame ' -Running PPSS based on a configuration file. +Extended usage: use --help -./ppss.sh standalone -C config.cfg - -Running PPSS on a client as part of a cluster. - -./ppss.sh node -d /somedir -c 'cp "$ITEM" /some/destination' -s 10.0.0.50 -u ppss -t -k ppss-key.key' }}} + A detailed explanation based on examples will follow. = How to use PPSS =