Difference between revisions of "Code"

From duply (simple duplicity) - a duplicity shell frontend
Jump to navigation Jump to search
m (Edso moved page Duply-code to Code)
 
Line 1: Line 1:
 +
{{DISPLAYTITLE:Sourcecode / SVN}}
 
<!--
 
<!--
 
=Include Test=
 
=Include Test=

Latest revision as of 14:00, 18 June 2020


duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/

SVN access

svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply

SVN Log

https://sourceforge.net/p/ftplicity/code/commit_browser

Latest Development Snapshot

mod time 2020-07-07

plain/text -> duply.sh

   1 #!/usr/bin/env bash
   2 #
   3 ################################################################################
   4 #  duply (grown out of ftplicity), is a shell front end to duplicity that      #
   5 #  simplifies the usage by managing settings for backup jobs in profiles.      #
   6 #  It supports executing multiple commands in a batch mode to enable single    #
   7 #  line cron entries and executes pre/post backup scripts.                     #
   8 #  Since version 1.5.0 all duplicity backends are supported. Hence the name    #
   9 #  changed from ftplicity to duply.                                            #
  10 #  See http://duply.net or http://ftplicity.sourceforge.net/ for more info.    #
  11 #  (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany            #
  12 #  (c) 2008-2019 Edgar Soldin (changes since version 1.3)                      #
  13 ################################################################################
  14 #  LICENSE:                                                                    #
  15 #  This program is licensed under GPLv2.                                       #
  16 #  Please read the accompanying license information in gpl-2.0.txt.            #
  17 ################################################################################
  18 #  TODO/IDEAS/KNOWN PROBLEMS:
  19 #  - possibility to restore time frames (incl. deleted files)
  20 #    realizable by listing each backup and restore from 
  21 #    oldest to the newest, problem: not performant
  22 #  - search file in all backups function and show available
  23 #    versions with backups date (list old avail since 0.6.06)
  24 #  - edit profile opens conf file in vi 
  25 #  - implement log-fd interpretation
  26 #  - add a duplicity option check against the options pending 
  27 #    deprecation since 0.5.10 namely --time-separator
  28 #                               --short-filenames
  29 #                              --old-filenames
  30 #  - add 'exclude_<command>' list usage e.g. exclude_verify
  31 #  - featreq 25: a download/install duplicity option
  32 #  - hint on install software if a piece is missing
  33 #  - import/export profile from/to .tgz function !!!
  34 #
  35 #  CHANGELOG:
  36 #  2.2.2 (24.02.2020)
  37 #  - bugfix 120: Failures in "Autoset trust of key" during restore 
  38 #    because of gpg2.2 fingerprint output change
  39 #
  40 #  2.2.1 (22.01.2020)
  41 #  - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  42 #  - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  43 #  - fix abbreviation spelling of 'e.g.'
  44 #
  45 #  2.2 (30.12.2018)
  46 #  - featreq 44: implement grouping for batch commands
  47 #      new separators are [] (square brackets) or groupIn/groupOut
  48 #      command 'backup' translates now to [pre_bkp_post] to be skipped as
  49 #      one block in case a condition was set in the batch instruction
  50 #
  51 #  2.1 (23.07.2018)
  52 #  - be more verbose when duplicity version detection fails
  53 #  - using info shows python binary's path for easier identification now
  54 #  - reworked python interpreter handling, it's either
  55 #      configured per PYTHON var
  56 #      unconfigured, parsed from duplicity shebang
  57 #      or set to current duplicity default 'python2' (was 'python' until now)
  58 #  - do not quotewrap strings because of slashes (e.g. paths) anymore
  59 #  - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  60 #
  61 #  2.0.4 (20.02.2018)
  62 #  - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  63 #  - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  64 #  - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  65 #  - bugfix debian 882159: duply: occasionally shows negative runtimes
  66 #
  67 #  2.0.3 (29.08.2017)
  68 #  - bugfix: "line 2231: CMDS: bad array subscript"
  69 #  - bugfix 112: "env: illegal option -- u" on MacOSX
  70 #
  71 #  2.0.2 (23.05.2017)
  72 #  - bugfix: never insert creds into file:// targets
  73 #  - bugfix: avail profiles hint sometimes shortend the names by one char
  74 #  - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  75 #  - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  76 #  - bugfix 97: Unknown command should be ERROR, not WARNING 
  77 #
  78 #  2.0.1 (16.11.2016)
  79 #  - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
  80 #
  81 #  2.0 (27.10.2016)
  82 #  made this a major version change, as we broke backward compatibility anyway
  83 #  (see last change in v1.10). got complaints that rightfully pointed out
  84 #  that should only come w/ a major version change. so, here we go ;)
  85 #  if your backend stops working w/ this version create a new profile and
  86 #  export the env vars needed as described in the comments of the conf file
  87 #  directly above the SOURCE setting.
  88 #  Changes:
  89 #  - making sure multi spaces in TARGET survive awk processing
  90 #  - new env var PROFILE exported to scripts 
  91 #  - fix 102: expose a unique timestamp variable for pre/post scripts
  92 #    actually a featreq. exporting RUN_START nanosec unix timestamp
  93 #  - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
  94 #  - fix 96: duply cannot handle two consecutive spaces in paths
  95 #
  96 #  1.11.3 (29.5.2016)
  97 #  - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
  98 #
  99 #  1.11.2 (11.2.2016)
 100 #  - fix "gpg: unsafe" version print out 
 101 #  - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
 102 #  - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional 
 103 #    documentation about needed env vars to template conf file
 104 #
 105 #  1.11.1 (18.12.2015)
 106 #  - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
 107 #  - reverted duply's default PYTHON to 'python'
 108 #
 109 #  1.11 (24.11.2015)
 110 #  - remove obsolete --ssh-askpass routine
 111 #  - add PYTHON conf var to allow global override of used python interpreter
 112 #  - enforced usage of "python2" in PATH as default interpreter for internal
 113 #    use _and_ to run duplicity (setup.py changed the shebang to the fixed
 114 #    path /usr/bin/python until 0.7.05, which we circumvent this way)
 115 #  - featreq 36: support gpg-connect-agent as a means to detect if an agent is 
 116 #    running (thx Thomas Harning Jr.), used gpg-agent for detection though
 117 #  - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
 118 #  - key export routine respects gpg-agent usage now
 119 #
 120 #  1.10.1 (19.8.2015)
 121 #  - bugfix 86: Duply+Swift outputs warning
 122 #  - bugfix 87: Swift fails without BACKEND_URL
 123 #
 124 #  1.10 (31.7.2015)
 125 #  - featreq 37: busybox issues - fix awk, grep version detection,
 126 #    fix grep failure because --color=never switch is unsupported
 127 #    (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
 128 #  - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
 129 #    (thx Joachim Wiedorn, also for maintaining the debian package)
 130 #  - implemented base-/dirname as bash functions
 131 #  - featreq 31 " Support for duplicity Azure backend " - ignored a 
 132 #    contributed patch by Scott McKenzie and instead opted for removing almost
 133 #    all code that deals with special env vars required by backends.
 134 #    adding and modifying these results in too much overhead so i dropped this
 135 #    feature. the future alternative for users is to consult the duplicity 
 136 #    manpage and add the needed export definitions to the conf file.
 137 #    appended a commented example to the template conf below the auth section.
 138 #
 139 #  1.9.2 (21.6.2015)
 140 #  - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
 141 #  - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
 142 #  - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
 143 #  - added GPG conf var (see conf template for details)
 144 #  - added grep version output as it is an integral needed binary
 145 #  - added PYTHONPATH printout in version output
 146 #
 147 #  1.9.1 (13.10.2014)
 148 #  - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
 149 #  - bugfix: CMD_PREV contained command even if it was skipped
 150 #
 151 #  1.9.0 (24.8.2014)
 152 #  - bugfix: env vars were not exported when external script was executable
 153 #  - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) 
 154 #    see gpg manpage, section "How to specify a user ID"
 155 #    let gpg complain when the delivered values are invalid for whatever reason
 156 #  - started to rework tmp space checking, exposed folder & writable check
 157 #    TODO: reimplement enough file space available checking
 158 #
 159 #  1.8.0 (13.7.2014)
 160 #  - add command verifyPath to expose 'verify --file-to-restore' action
 161 #  - add time parameter support to verify command
 162 #  - add section time formats to usage output 
 163 #
 164 #  1.7.4 (24.6.2014)
 165 #  - remove ubuntu one support, service is discontinued
 166 #  - featreq 31: add authenticated swift (contributed by Justus Seifert)
 167 #
 168 #  1.7.3 (3.4.2014)
 169 #  - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
 170 #
 171 #  1.7.2 (1.4.2014 "April,April")
 172 #  - bugfix: debian Bug#743190 "duply no longer allows restoration without 
 173 #     gpg passphrase in conf file"
 174 #     GPG_AGENT_INFO env var is now needed to trigger --use-agent
 175 #  - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
 176 #
 177 #  1.7.1 (30.3.2014)
 178 #  - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to 
 179 #     incompatibility with new minus batch separator 
 180 #
 181 #  1.7.0 (20.3.2014)
 182 #  - disabled gpg key id plausibility check, too many valid possibilities
 183 #  - featreq 7 "Halt if precondition fails":
 184 #     added and(+), or(-) batch command(separator) support
 185 #  - featreq 26 "pre/post script with shebang line": 
 186 #     if a script is flagged executable it's executed in a subshell 
 187 #     now as opposed to sourced to bash, which is the default
 188 #  - bugfix: do not check if dpbx, swift credentials are set anymore 
 189 #  - bugfix: properly escape profile name, archdir if used as arguments
 190 #  - add DUPL_PRECMD conf setting for use with e.g. trickle
 191 #
 192 #  1.6.0 (1.1.2014)
 193 #  - support gs backend
 194 #  - support dropbox backend
 195 #  - add gpg-agent support to gpg test routines
 196 #  - autoenable --use-agent if passwords were not defined in config
 197 #  - GPG_OPTS are now honored everywhere, keyrings or complete gpg
 198 #    homedir can thus be configured to be located anywhere
 199 #  - always import both secret and public key if avail from config profile
 200 #  - new explanatory comments in initial exclude file
 201 #  - bugfix 7: Duply only imports one key at a time 
 202 #
 203 #  1.5.11 (19.07.2013)
 204 #  - purge-incr command for remove-all-inc-of-but-n-full feature added
 205 #    patch provided by Moritz Augsburger, thanks!
 206 #  - documented version command in man page
 207 #
 208 #  1.5.10 (26.03.2013)
 209 #  - minor indent and documentation fixes
 210 #  - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
 211 #  - bugfix: fix url_decoding generally and for python3
 212 #  - bugfix 3609075: wrong script results in status line (thx David Epping)
 213 #
 214 #  1.5.9 (22.11.2012)
 215 #  - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
 216 #  - restore/fetch now also ignores --include* or --exclude='foobar' 
 217 #
 218 #  1.5.8 (26.10.2012)
 219 #  - bugfix 3575487: implement proper cloud files support
 220 #
 221 #  1.5.7 (10.06.2012)
 222 #  - bugfix 3531450: Cannot use space in target URL (file:///) anymore
 223 #
 224 #  1.5.6 (24.5.2012)
 225 #  - commands purge, purge-full have no default value anymore for security 
 226 #    reasons; instead max value can be given via cmd line or must be set
 227 #    in profile; else an error is shown.
 228 #  - minor man page modifications
 229 #
 230 #  versioning scheme will be simplified to [major].[minor].[patch] version
 231 #  with the next version raise
 232 #
 233 #  1.5.5.5 (4.2.2012)
 234 #  - bugfix 3479605: SEL context confused profile folder's permission check
 235 #  - colon ':' in url passphrase got ignored, added python driven url_decoding
 236 #    for user & pass to better process special chars
 237 #
 238 #  1.5.5.4 (16.10.2011)
 239 #  - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
 240 #  - add support for separate sign passphrase (needs duplicity 0.6.14+)
 241 #
 242 #  1.5.5.3 (1.10.2011)
 243 #  - bugfix 3416690: preview threw echo1 error
 244 #  - fix unknown cmds error usage & friends if more than 2 params were given
 245 #
 246 #  1.5.5.2 (23.9.2011)
 247 #  - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
 248 #  - bugfix: mawk does not support \W and did not split multikey definitions
 249 #  - all parameters should survive  single (') and double (") quotes now
 250 #
 251 #  1.5.5.1 (7.6.2011)
 252 #  - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
 253 #  - bugfix 3312208: signing detection broke symmetric gpg test routine
 254 #
 255 #  1.5.5 (2.5.2011)
 256 #  - bugfix: fetch problem with space char in path, escape all params 
 257 #    containing non word chars
 258 #  - list available profiles, if given profile cannot be found
 259 #  - added --use-agent configuration hint
 260 #  - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke 
 261 #    fetch/restore
 262 #  - version command now prints out 'using installed' info
 263 #  - featreq 3166169: autotrust imported keys, based on code submitted by 
 264 #    Martin Ellis - imported keys are now automagically trusted ultimately 
 265 #  - new txt2man feature to create manpages for package maintainers
 266 #
 267 #  1.5.4.2 (6.1.2011)
 268 #  - new command changelog
 269 #  - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
 270 #  - bugfix: freebsd awk hangs on 'awk -W version' 
 271 #  - bugfix 3150244: mawk does not know '--version'
 272 #  - minor help text improvements
 273 #  - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
 274 #
 275 #  1.5.4.1 (4.12.2010)
 276 #  - output awk, python, bash version now in prolog
 277 #  - shebang uses /usr/bin/env now for freebsd compatibility, 
 278 #    bash not in /bin/bash 
 279 #  - new --disable-encryption parameter, 
 280 #    to override profile encr settings for one run
 281 #  - added exclude-if-present setting to conf template
 282 #  - bug 3126972: GPG_PW only needed for signing/symmetric encryption 
 283 #    (even though duplicity still needs it)
 284 #
 285 #  1.5.4 (15.11.2010)
 286 #  - as of 1.5.3 already, new ARCH_DIR config option
 287 #  - multiple key support
 288 #  - ftplicity-Feature Requests-2994929: separate encryption and signing key
 289 #  - key signing of symmetric encryption possible (duplicity patch committed)
 290 #  - gpg tests disable switch
 291 #  - gpg tests now previewable and more intelligent
 292 #
 293 #  1.5.3 (1.11.2010)
 294 #  - bugfix 3056628: improve busybox compatibility, grep did not have -m param
 295 #  - bugfix 2995408: allow empty password for PGP key
 296 #  - bugfix 2996459: Duply erroneously escapes '-' symbol in username
 297 #  - url_encode function is now pythonized
 298 #  - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
 299 #  - feature 3059262: Make pre and post aware of parameters, 
 300 #                     internal parameters + CMD of pre or post 
 301 #
 302 #  1.5.2.3 (16.4.2010)
 303 #  - bugfix: date again, should now work virtually anywhere
 304 #
 305 #  1.5.2.2 (3.4.2010)
 306 #  - minor bugfix: duplicity 0.6.8b version string now parsable
 307 #  - added INSTALL.txt
 308 #
 309 #  1.5.2.1 (23.3.2010)
 310 #  - bugfix: date formatting is awked now and should work on all platforms
 311 #
 312 #  1.5.2 (2.3.2010)
 313 #  - bugfix: errors print to STD_ERR now, failed tasks print an error message
 314 #  - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
 315 #  - simplified & cleaned profileless commands, removed second instance
 316 #  - generalized separator time routines
 317 #  - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
 318 #  - minor fixes
 319 #
 320 #  1.5.1.5 (5.2.2010)
 321 #  - bugfix: added special handling of credentials for rsync, imap(s)
 322 #
 323 #  1.5.1.4 (7.1.2010)
 324 #  - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
 325 #  - check if ncftp binary is available if url protocol is ftp
 326 #  - bugfix: duplicity output is now printed to screen directly to resolve
 327 #            'mem alloc problem' bug report
 328 #  - bugfix: passwords will not be in the url anymore to solve the 'duply shows
 329 #            sensitive data in process listing' bug report
 330 #
 331 #  1.5.1.3 (24.12.2009) 'merry xmas'
 332 #  - bugfix: gpg pass now apostrophed to allow space and friends
 333 #  - bugfix: credentials are now url encoded to allow special chars in them
 334 #            a note about url encoding has been added to the conf template
 335 #
 336 #  1.5.1.2 (1.11.2009)
 337 #  - bugfix: open parenthesis in password broke duplicity execution
 338 #  - bugfix: ssh/scp backend does not always need credentials e.g. key auth
 339 #
 340 #  1.5.1.1 (21.09.2009)
 341 #  - bugfix: fixed s3[+http] TARGET_PASS not needed routine
 342 #  - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
 343 #    see https://sourceforge.net/tracker/index.php?func=detail&
 344 #                aid=2864410&group_id=217745&atid=1041147
 345 #
 346 #  1.5.1 (21.09.2009) - duply (fka. ftplicity)
 347 #  - first things first: ftplicity (being able to support all backends since 
 348 #    some time) will be called duply (fka. ftplicity) from now on. The addendum
 349 #    is for the time being to circumvent confusion.
 350 #  - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity 
 351 #            command failed
 352 #  - s3[+http] now supported natively by translating user/pass to access_key/
 353 #    secret_key environment variables needed by duplicity s3 boto backend 
 354 #  - bugfix: additional output lines do not confuse version check anymore
 355 #  - list command supports now age parameter (patch by stefan on feature 
 356 #    request tracker)
 357 #  - bugfix: option/param pairs are now correctly passed on to duplicity
 358 #  - bugfix: s3[+http] needs no TARGET_PASS if command is read only
 359 #
 360 #  1.5.0.2 (31.07.1009)
 361 #  - bugfix: insert password in target url didn't work with debian mawk
 362 #            related to previous bug report
 363 #
 364 #  1.5.0.1 (23.07.2009)
 365 #  - bugfix: gawk gensub dependency raised an error on debian's default mawk
 366 #            replaced with match/substr command combination (bug report)
 367 #            https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
 368 #            group_id=217745
 369 #
 370 #  1.5.0 (01.07.2009)
 371 #  - removed ftp limitation, all duplicity backends should work now
 372 #  - bugfix: date for separator failed on openwrt busybox date, added a 
 373 #    detecting workaround, milliseconds are not available w/ busybox date
 374 #
 375 #  1.4.2.1 (14.05.2009)
 376 #  - bugfix: free temp space detection failed with lvm, fixed awk parse routine
 377 #
 378 #  1.4.2 (22.04.2009)
 379 #  - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
 380 #    armored ascii nature, the id helps if the key is switched for some reason
 381 #    im/export routines are updated accordingly (import is backward compatible 
 382 #    to the old profile/gpgkey files)         
 383 #  - profile argument is treated as path if it contains slashes 
 384 #    (for details see usage)
 385 #  - non-ftplicity options (all but --preview currently) are now passed 
 386 #    on to duplicity 
 387 #  - removed need for stat in secure_conf, it is ls based now
 388 #  - added profile folder readable check
 389 #  - added gpg version & home info output
 390 #  - awk utility availability is now checked, because it was mandatory already
 391 #  - tmp space is now checked on writability and space requirement
 392 #    test fails on less than 25MB or configured $VOLSIZE, 
 393 #    test warns if there is less than two times $VOLSIZE because 
 394 #    that's required for --asynchronous-upload option  
 395 #  - gpg functionality is tested now before executing duplicity 
 396 #    test drive contains encryption, decryption, comparison, cleanup
 397 #    this is meant to detect non trusted or other gpg errors early
 398 #  - added possibility of doing symmetric encryption with duplicity
 399 #    set GPG_KEY="" or simply comment it out
 400 #  - added hints in config template on the depreciation of 
 401 #    --short-filenames, --time-separator duplicity options
 402 #
 403 #  new versioning scheme 1.4.2b => 1.4.2, 
 404 #  beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
 405 #  to the first bug fixing version and 1.4.2.2 to the second and so on
 406 #  also the releases will now have a release date formatted (Day.Month.Year)
 407 #
 408 #  1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
 409 #            named exactly as the profile if existing in executing dir
 410 #          - improved plausibility checking of config and profile folder
 411 #          - secure_conf only acts if needed and prints a warning now
 412 #
 413 #  1.4.1b  - introduce status (duplicity collection-status) command
 414 #          - pre/post script output printed always now, not only on errors
 415 #          - new config parameter GPG_OPTS to pass gpg options
 416 #            added examples & comments to profile template conf
 417 #          - reworked separator times, added duration display
 418 #          - added --preview switch, to preview generated command lines
 419 #          - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
 420 #            profiles because they have reasonable defaults now if not set
 421 #
 422 #  1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
 423 #            therefore backup translates to pre_bkp_post now
 424 #          - bugfix: new command bkp, which represents duplicity's 
 425 #            default action (incr or full if full_if_older matches
 426 #            or no earlier backup chain is found)
 427 #
 428 #  new versioning scheme 1.4 => 1.4.0, added new minor revision number
 429 #  this is meant to slow down the rapid version growing but still keep 
 430 #  versions cleanly separated.
 431 #  only additional features will raise the new minor revision number. 
 432 #  all releases start as beta, each bugfix release will raise the beta 
 433 #  count, usually new features arrive before a version 'ripes' to stable
 434 #    
 435 #  1.4.0b
 436 #    1.4b  - added startup info on version, time, selected profile
 437 #          - added time output to separation lines
 438 #          - introduced: command purge-full implements duplicity's 
 439 #            remove-all-but-n-full functionality (patch by unknown),
 440 #            uses config variable $MAX_FULL_BACKUPS (default = 1)
 441 #          - purge config var $MAX_AGE defaults to 1M (month) now 
 442 #          - command full does not execute pre/post anymore
 443 #            use batch command pre_full_post if needed 
 444 #          - introduced batch mode cmd1_cmd2_etc
 445 #            (in turn removed the bvp command)
 446 #          - unknown/undefined command issues a warning/error now
 447 #          - bugfix: version check works with 0.4.2 and older now
 448 #    1.3b3 - introduced pre/post commands to execute/debug scripts
 449 #          - introduced bvp (backup, verify, purge)
 450 #          - bugfix: removed need for awk gensub, now mawk compatible
 451 #    1.3b2 - removed pre/post need executable bit set 
 452 #          - profiles now under ~/.ftplicity as folders
 453 #          - root can keep profiles in /etc/ftplicity, folder must be
 454 #            created by hand, existing profiles must be moved there
 455 #          - removed ftplicity in path requirement
 456 #          - bugfix: bash < v.3 did not know '=~'
 457 #          - bugfix: purge works again 
 458 #    1.3   - introduces multiple profiles support
 459 #          - modified some script errors/docs
 460 #          - reordered gpg key check import routine
 461 #          - added 'gpg key id not set' check
 462 #          - added error_gpg (adds how to setup gpg key howto)
 463 #          - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
 464 #          - duplicity_version_check routine introduced
 465 #          - added time separator, shortnames, volsize, full_if_older 
 466 #            duplicity options to config file (inspired by stevie 
 467 #            from http://weareroot.de) 
 468 #    1.1.1 - bugfix: encryption reactivated
 469 #    1.1   - introduced config directory
 470 #    1.0   - first release
 471 ################################################################################
 472 
 473 # utility functions overriding binaries
 474 
 475 # wrap grep to override possible env set GREP_OPTIONS=--color=always
 476 function grep {
 477   command env "GREP_OPTIONS=" grep "$@"
 478 }
 479 
 480 # implement basename in plain bash
 481 function basename {
 482   local stripped="${1%/}"
 483   echo "${stripped##*/}"
 484 }
 485 
 486 # implement dirname in plain bash
 487 function dirname {
 488   echo ${1%/*}
 489 }
 490 
 491 # implement basic which in plain bash
 492 function which {
 493   type -p "$@"
 494 }
 495 
 496 # check availability of executables via file name or file paths
 497 function lookup {
 498   local bin="$1"
 499   # look for file names in path via bash hash OR 
 500   # look for executables at given relative/absolute location
 501   ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
 502 }
 503 
 504 # the python binary to use, exit code 0 when configured, else 1
 505 function python_binary {
 506   # if unset, parse from duplicity shebang
 507   if ! var_isset 'PYTHON'; then
 508     duplicity_python_binary_parse;
 509     echo $DUPL_PYTHON_BIN;
 510     return 1;
 511   else
 512     # tell if PYTHON was configured manually
 513     echo $PYTHON;
 514     return 0
 515   fi
 516 }
 517 
 518 # important definitions #######################################################
 519 
 520 ME_LONG="$0"
 521 ME="$(basename $0)"
 522 ME_NAME="${ME%%.*}"
 523 ME_VERSION="2.2.2"
 524 ME_WEBSITE="http://duply.net"
 525 
 526 # default config values
 527 DEFAULT_SOURCE='/path/of/source'
 528 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
 529 DEFAULT_TARGET_USER='_backend_username_'
 530 DEFAULT_TARGET_PASS='_backend_password_'
 531 DEFAULT_GPG='gpg'
 532 DEFAULT_GPG_KEY='_KEY_ID_'
 533 DEFAULT_GPG_PW='_GPG_PASSWORD_'
 534 DEFAULT_PYTHON='python2'
 535 
 536 # function definitions ##########################
 537 
 538 function set_config { # sets global config vars
 539   local CONFHOME_COMPAT="$HOME/.ftplicity"
 540   local CONFHOME="$HOME/.duply"
 541   local CONFHOME_ETC_COMPAT="/etc/ftplicity"
 542   local CONFHOME_ETC="/etc/duply"
 543 
 544   # confdir can be delivered as path (must contain /)
 545   if [ `echo $FTPLCFG | grep /` ] ; then 
 546     CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
 547               ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
 548                && echo $(pwd)/${FTPLCFG} ) || \
 549               echo ${FTPLCFG})          
 550   # or DEFAULT in home/.duply folder (NEW)
 551   elif [ -d "${CONFHOME}" ]; then
 552     CONFDIR="${CONFHOME}/${FTPLCFG}"
 553   # or in home/.ftplicity folder (OLD)
 554   elif [ -d "${CONFHOME_COMPAT}" ]; then
 555     CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
 556     warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
 557   # root can put profiles under /etc/duply (NEW) if path exists
 558   elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
 559     CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
 560   # root can keep profiles under /etc/ftplicity (OLD) if path exists
 561   elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
 562     CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
 563     warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
 564   # hmm no profile folder there, then use default for error later
 565   else
 566     CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
 567   fi
 568 
 569   # remove trailing slash, get profile name etc.
 570   CONFDIR="${CONFDIR%/}"
 571   PROFILE="${CONFDIR##*/}"
 572   CONF="$CONFDIR/conf"
 573   PRE="$CONFDIR/pre"
 574   POST="$CONFDIR/post"
 575   EXCLUDE="$CONFDIR/exclude"
 576   KEYFILE="$CONFDIR/gpgkey.asc"
 577 }
 578 
 579 function version_info { # print version information
 580   cat <<END
 581   $ME_NAME version $ME_VERSION
 582   ($ME_WEBSITE)
 583 END
 584 }
 585 
 586 function version_info_using { 
 587   cat <<END
 588 $(version_info)
 589 
 590   $(using_info)
 591 END
 592 }
 593 
 594 function using_info {
 595   # init needed vars into global name space
 596   lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
 597   local NOTFOUND="INVALID"
 598   local AWK_VERSION GREP_VERSION PYTHON_RUNNER
 599   # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
 600   AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 601   GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 602   PYTHON_RUNNER=$(python_binary)
 603   local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
 604   local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
 605   local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
 606   # print out
 607   echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
 608 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
 609 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
 610 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
 611 }
 612 
 613 function usage_info { # print usage information
 614 
 615   cat <<USAGE_EOF
 616 VERSION:
 617 $(version_info)
 618   
 619 DESCRIPTION: 
 620   Duply deals as a wrapper for the mighty duplicity magic.
 621   It simplifies running duplicity with cron or on command line by:
 622 
 623     - keeping recurring settings in profiles per backup job
 624     - enabling batch operations e.g. backup_verify+purge
 625     - executing pre/post scripts (different actions possible 
 626       depending on previous or next command or it's exit status)
 627     - precondition checking for flawless duplicity operation
 628 
 629   For each backup job one configuration profile must be created.
 630   The profile folder will be stored under '~/.${ME_NAME}/<profile>'
 631   (where ~ is the current users home directory).
 632   Hint:  
 633    If the folder '/etc/${ME_NAME}' exists, the profiles for the super
 634    user root will be searched & created there.
 635 
 636 USAGE:
 637   first time usage (profile creation):  
 638     $ME <profile> create
 639 
 640   general usage in single or batch mode (see EXAMPLES):  
 641     $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
 642 
 643   For batches the conditional separators can also be written as pseudo commands
 644   and(+), or(-). See SEPARATORS for details.
 645 
 646   Non $ME options are passed on to duplicity (see OPTIONS).
 647   All conf parameters can also be defined in the environment instead.
 648 
 649 PROFILE:
 650   Indicated by a path or a profile name (<profile>), which is resolved 
 651   to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
 652 
 653   Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
 654   the folder manually before running $ME_NAME as superuser.
 655   Note:  
 656     Already existing profiles in root's home folder will cease to work
 657     unless they are moved to the new location manually.
 658 
 659   example 1:   $ME humbug backup
 660 
 661   Alternatively a _path_ might be used e.g. useful for quick testing, 
 662   restoring or exotic locations. Shell expansion should work as usual.
 663   Hint:  
 664     The path must contain at least one path separator '/', 
 665     e.g. './test' instead of only 'test'.
 666 
 667   example 2:   $ME ~/.${ME_NAME}/humbug backup
 668 
 669 SEPARATORS:
 670   _ (underscore)  
 671              neutral separator
 672   + (plus sign), _and_  
 673              conditional AND
 674              the next command will only be executed if the previous succeeded
 675   - (minus sign), _or_  
 676              conditional OR
 677              the next command will only be executed if the previous failed
 678   [] (square brackets), _groupIn_/_groupOut_  
 679              enables grouping of commands
 680 
 681    example:  
 682     'pre+[bkp-verify]_post' translates to
 683     'pre_and_groupIn_bkp_or_verify_groupOut_post'
 684 
 685 COMMANDS:
 686   usage      get usage help text
 687 
 688   and/or/groupIn/groupOut  
 689              pseudo commands used in batches (see SEPARATORS above)
 690 
 691   create     creates a configuration profile
 692   backup     backup with pre/post script execution (batch: [pre_bkp_post]),
 693               full (if full_if_older matches or no earlier backup is found)
 694               incremental (in all other cases)
 695   pre/post   execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
 696   bkp        as above but without executing pre/post scripts
 697   full       force full backup
 698   incr       force incremental backup
 699   list [<age>]  
 700              list all files in backup (as it was at <age>, default: now)
 701   status     prints backup sets and chains currently in repository
 702   verify [<age>] [--compare-data]  
 703              list files changed, since age if given
 704   verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]  
 705              list changes of a file or folder path in backup compared to a
 706              local path, since age if given
 707   restore <target_path> [<age>]  
 708              restore the complete backup to <target_path> [as it was at <age>]
 709   fetch <src_path> <target_path> [<age>]  
 710              fetch single file/folder from backup [as it was at <age>]
 711   purge [<max_age>] [--force]  
 712              list outdated backup files (older than \$MAX_AGE)
 713               [use --force to actually delete these files]
 714   purgeFull [<max_full_backups>] [--force]  
 715              list outdated backup files (\$MAX_FULL_BACKUPS being the number of
 716              full backups and associated incrementals to keep, counting in 
 717              reverse chronological order)
 718               [use --force to actually delete these files]
 719   purgeIncr [<max_fulls_with_incrs>] [--force]  
 720              list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being 
 721              the number of full backups which associated incrementals will be
 722              kept, counting in reverse chronological order) 
 723               [use --force to actually delete these files]
 724   cleanup [--force]  
 725              list broken backup chain files archives (e.g. after unfinished run)
 726               [use --force to actually delete these files]
 727 
 728   changelog  print changelog / todo list
 729   txt2man    feature for package maintainers - create a manpage based on the 
 730              usage output. download txt2man from http://mvertes.free.fr/, put 
 731              it in the PATH and run '$ME txt2man' to create a man page.
 732   version    show version information of $ME_NAME and needed programs
 733 
 734 OPTIONS:
 735   --force    passed to duplicity (see commands: 
 736              purge, purgeFull, purgeIncr, cleanup)
 737   --preview  do nothing but print out generated duplicity command lines
 738   --disable-encryption  
 739              disable encryption, overrides profile settings
 740 
 741 TIME FORMATS:
 742   For all time related parameters like age, max_age etc.
 743   Refer to the duplicity manpage for all available formats. Here some examples:
 744     2002-01-25T07:00:00+02:00 (full date time format string)
 745     2002/3/5 (date string YYYY/MM/DD)
 746     12D (interval, 12 days ago)
 747     1h78m (interval, 1 hour 78 minutes ago)
 748 
 749 PRE/POST SCRIPTS:
 750   Some useful internal duply variables are exported to the scripts.
 751 
 752     PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>, 
 753     GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
 754     CMD_<PREV|NEXT> (previous/next command), 
 755     CND_<PREV|NEXT> (condition before/after)
 756 
 757   The CMD_* variables were introduced to allow different actions according to 
 758   the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post' 
 759   will call the pre script two times, with CMD_NEXT variable set to 'bkp' 
 760   on the first and to 'verify' on the second run.
 761   CMD_ERR holds the exit code of the CMD_PREV .
 762 
 763 EXAMPLES:
 764   create profile 'humbug':  
 765     $ME humbug create (don't forget to edit this new conf file)
 766   backup 'humbug' now:  
 767     $ME humbug backup
 768   list available backup sets of profile 'humbug':  
 769     $ME humbug status
 770   list and delete outdated backups of 'humbug':  
 771     $ME humbug purge --force
 772   restore latest backup of 'humbug' to /mnt/restore:  
 773     $ME humbug restore /mnt/restore
 774   restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:  
 775     $ME humbug fetch etc/passwd /root/pw 4D
 776     (see "duplicity manpage", section TIME FORMATS)
 777   a one line batch job on 'humbug' for cron execution:  
 778     $ME humbug backup_verify_purge --force
 779   batch job to run a full backup with pre/post scripts:  
 780     $ME humbug pre_full_post
 781 
 782 FILES:
 783   in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
 784   conf             profile configuration file
 785   pre,post         pre/post scripts (see above for details)
 786   gpgkey.*.asc     exported GPG key files
 787   exclude          a globbing list of included or excluded files/folders
 788                    (see "duplicity manpage", section FILE SELECTION)
 789 
 790 $(hint_profile)
 791 
 792 SEE ALSO:
 793   duplicity man page:
 794     duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
 795 USAGE_EOF
 796 }
 797 
 798 # to check call 'duply txt2man | man -l -'
 799 function usage_txt2man {
 800   usage_info | \
 801   awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
 802   txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
 803 }
 804 
 805 function changelog {
 806   cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#(  )?/,"",$0);print}'
 807 }
 808 
 809 function create_config {
 810   if [ ! -d "$CONFDIR" ] ; then
 811     mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
 812   # create initial config file
 813     cat <<EOF >"$CONF"
 814 # gpg encryption settings, simple settings:
 815 #  GPG_KEY='disabled' - disables encryption alltogether
 816 #  GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
 817 #   sign if secret key of key1 is available use GPG_PW for sign & decrypt
 818 #  Note: you can specify keys via all methods described in gpg manpage,
 819 #        section "How to specify a user ID", escape commas (,) via backslash (\)
 820 #        e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
 821 #        as they are used to separate the entries
 822 #  GPG_PW='passphrase' - symmetric encryption using passphrase only
 823 GPG_KEY='${DEFAULT_GPG_KEY}'
 824 GPG_PW='${DEFAULT_GPG_PW}'
 825 # gpg encryption settings in detail (extended settings)
 826 #  the above settings translate to the following more specific settings
 827 #  GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
 828 #  GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
 829 #  GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
 830 #   encryption. If you want to deliver different passphrases for e.g. 
 831 #   several keys or symmetric encryption plus key signing you can use
 832 #   gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
 833 #   also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage 
 834 # notes on en/decryption
 835 #  private key and passphrase will only be needed for decryption or signing.
 836 #  decryption happens on restore and incrementals (compare archdir contents).
 837 #  for security reasons it makes sense to separate the signing key from the
 838 #  encryption keys. https://answers.launchpad.net/duplicity/+question/107216
 839 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
 840 #GPG_KEY_SIGN='<prvkey>'
 841 # set if signing key passphrase differs from encryption (key) passphrase
 842 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
 843 #GPG_PW_SIGN='<signpass>'
 844 
 845 # uncomment and set a file path or name force duply to use this gpg executable
 846 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
 847 #GPG='/usr/local/gpg-2.1/bin/gpg'
 848 
 849 # gpg options passed from duplicity to gpg process (default='')
 850 # e.g. "--trust-model pgp|classic|direct|always" 
 851 #   or "--compress-algo=bzip2 --bzip2-compress-level=9"
 852 #   or "--personal-cipher-preferences AES256,AES192,AES..."
 853 #   or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
 854 #   or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
 855 #      also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
 856 #GPG_OPTS=''
 857 
 858 # disable preliminary tests with the following setting
 859 #GPG_TEST='disabled'
 860 
 861 # backend, credentials & location of the backup target (URL-Format)
 862 # generic syntax is
 863 #   scheme://[user[:password]@]host[:port]/[/]path
 864 # e.g.
 865 #   sftp://bob:secret@backupserver.com//home/bob/dupbkp
 866 # for details and available backends see duplicity manpage, section URL Format
 867 #   http://duplicity.nongnu.org/duplicity.1.html#sect7
 868 # BE AWARE:
 869 #   some backends (cloudfiles, S3 etc.) need additional env vars to be set to
 870 #   work properly, read after the TARGET definition for more details.
 871 # ATTENTION:
 872 #   characters other than A-Za-z0-9.-_.~ in the URL have to be
 873 #   replaced by their url encoded pendants, see
 874 #     http://en.wikipedia.org/wiki/Url_encoding
 875 #   if you define the credentials as TARGET_USER, TARGET_PASS below $ME
 876 #   will try to url_encode them for you if the need arises.
 877 TARGET='${DEFAULT_TARGET}'
 878 # optionally the username/password can be defined as extra variables
 879 # setting them here _and_ in TARGET results in an error
 880 # ATTENTION:
 881 #   there are backends that do not support the user/pass auth scheme.
 882 #   prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
 883 #   duplicity manpage. usually there is a NOTE section explaining if and which
 884 #   env vars should be set.
 885 #TARGET_USER='${DEFAULT_TARGET_USER}'
 886 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
 887 # e.g. for cloud files backend it might look like this (uncomment for use!)
 888 #export CLOUDFILES_USERNAME='someuser'
 889 #export CLOUDFILES_APIKEY='somekey'
 890 #export CLOUDFILES_AUTHURL ='someurl'
 891 # the following is an incomplete list (<backend>: comma separated env vars list)
 892 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
 893 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
 894 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
 895 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
 896 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
 897 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
 898 #        SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
 899 
 900 # base directory to backup
 901 SOURCE='${DEFAULT_SOURCE}'
 902 
 903 # a command that runs duplicity e.g. 
 904 #  shape bandwidth use via trickle
 905 #  "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
 906 #DUPL_PRECMD=""
 907 
 908 # override the used python interpreter, defaults to
 909 #  - parsed result of duplicity's shebang or 'python2'
 910 #   e.g. "python2" or "/usr/bin/python2.7"
 911 #PYTHON="python"
 912 
 913 # exclude folders containing exclusion file (since duplicity 0.5.14)
 914 # Uncomment the following two lines to enable this setting.
 915 #FILENAME='.duplicity-ignore'
 916 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
 917 
 918 # Time frame for old backups to keep, Used for the "purge" command.  
 919 # see duplicity man page, chapter TIME_FORMATS)
 920 #MAX_AGE=1M
 921 
 922 # Number of full backups to keep. Used for the "purgeFull" command. 
 923 # See duplicity man page, action "remove-all-but-n-full".
 924 #MAX_FULL_BACKUPS=1
 925 
 926 # Number of full backups for which incrementals will be kept for.
 927 # Used for the "purgeIncr" command.
 928 # See duplicity man page, action "remove-all-inc-of-but-n-full".
 929 #MAX_FULLS_WITH_INCRS=1
 930 
 931 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3) 
 932 # forces a full backup if last full backup reaches a specified age, for the 
 933 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
 934 # Uncomment the following two lines to enable this setting.
 935 #MAX_FULLBKP_AGE=1M
 936 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE " 
 937 
 938 # sets duplicity --volsize option (available since v0.4.3.RC7)
 939 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
 940 # VOLSIZE must be number of MB's to set the volume size to.
 941 # Uncomment the following two lines to enable this setting. 
 942 #VOLSIZE=50
 943 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
 944 
 945 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
 946 # default is 4, if not set
 947 #VERBOSITY=5
 948 
 949 # temporary file space. at least the size of the biggest file in backup
 950 # for a successful restoration process. (default is '/tmp', if not set)
 951 #TEMP_DIR=/tmp
 952 
 953 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
 954 # unencrypted meta data of the backup, enabling new incrementals without the 
 955 # need to decrypt backend metadata first. If empty or deleted somehow, the 
 956 # private key and it's password are needed.
 957 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
 958 #       big over time so you might want to put it not in the home dir.
 959 # default '~/.cache/duplicity/duply_<profile>/'
 960 # if set  '\${ARCH_DIR}/<profile>'
 961 #ARCH_DIR=/some/space/safe/.duply-cache
 962 
 963 # DEPRECATED setting
 964 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users 
 965 # to change the time separator from ':' to another character that will work 
 966 # on their system.  HINT: For Windows SMB shares, use --time-separator='_'.
 967 # NOTE: '-' is not valid as it conflicts with date separator.
 968 # ATTENTION: only use this with duplicity < 0.5.10, since then default file 
 969 #            naming is compatible and this option is pending depreciation 
 970 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
 971 
 972 # DEPRECATED setting
 973 # activates duplicity --short-filenames option, when uploading to a file
 974 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
 975 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
 976 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file 
 977 #            naming is compatible and this option is pending depreciation
 978 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
 979 
 980 # more duplicity command line options can be added in the following way
 981 # don't forget to leave a separating space char at the end
 982 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here " 
 983 
 984 EOF
 985 
 986 # create initial exclude file
 987     cat <<EOF >"$EXCLUDE"
 988 # although called exclude, this file is actually a globbing file list
 989 # duplicity accepts some globbing patterns, even including ones here
 990 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
 991 # - dir/bar/foo
 992 # + dir/bar
 993 # - **
 994 # for more details see duplicity manpage, section File Selection
 995 # http://duplicity.nongnu.org/duplicity.1.html#sect9
 996 
 997 EOF
 998 
 999   # Hints on first usage
1000   cat <<EOF
1001 
1002 Congratulations. You just created the profile '$FTPLCFG'.
1003 The initial config file has been created as 
1004 '$CONF'.
1005 You should now adjust this config file to your needs.
1006 
1007 $(hint_profile)
1008 
1009 EOF
1010 fi
1011 
1012 }
1013 
1014 # used in usage AND create_config
1015 function hint_profile {
1016   cat <<EOF
1017 IMPORTANT:
1018   Copy the _whole_ profile folder after the first backup to a safe place.
1019   It contains everything (duply related) needed to restore your backups. 
1020 
1021   Pay attention to (possibly later added) external files such as credentials
1022   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1023   (e.g. DPBX_ACCESS_TOKEN). 
1024   It is good policy to place those in the profile folder if possible at all.
1025     e.g. in case of 'multi://' target the config .json file
1026   Env vars should be added to duply profiles' conf file.
1027 
1028   Keep access to these files restricted as they contain information (gpg key,
1029   passphrases etc.) to access and modify your backups.
1030   
1031   Finally:
1032   You should attempt a restore from an unrelated host to be sure you really
1033   have everything needed for restoration.
1034 
1035   Repeat these steps after _all_ configuration changes. Some configuration 
1036   options are crucial for restoration.
1037 
1038 EOF
1039 }
1040 
1041 function separator {
1042   echo "--- $@ ---"
1043 }
1044 
1045 function inform {
1046   echo -e "\nINFO:\n\n$@\n"
1047 }
1048 
1049 function warning {
1050   echo -e "\nWARNING:\n\n$@\n"
1051 }
1052 
1053 function warning_oldhome {
1054   local old=$1 new=$2
1055   warning " ftplicity changed name to duply since you created your profiles.
1056   Please rename the old folder
1057   '$old'
1058   to
1059   '$new'
1060   and this warning will disappear.
1061   If you decide not to do so profiles will _only_ work from the old location."
1062 }
1063 
1064 function error_print {
1065   echo -e "$@" >&2
1066 }
1067 
1068 function error {
1069   error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1070   exit -1
1071 }
1072 
1073 function error_gpg {
1074   [ -n "$2" ] && local hint="\n  $2\n\n  "
1075   
1076   error "$1
1077 
1078 Hint${hint:+s}:
1079   ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1080   Don't forget the used _password_ as you will need it.
1081   When done enter the 8 digit id & the password in the profile conf file.
1082 
1083   The key id can be found doing a 'gpg --list-keys'. In the  example output 
1084   below the key id would be FFFFFFFF for the public key.
1085 
1086   pub   1024D/FFFFFFFF 2007-12-17
1087   uid                  duplicity
1088   sub   2048g/899FE27F 2007-12-17
1089 "
1090 }
1091 
1092 function error_gpg_key {
1093   local KEY_ID="$1"
1094   local KIND="$2"
1095   error_gpg "${KIND} gpg key '${KEY_ID}' cannot be found." \
1096 "Doublecheck if the above key is listed by 'gpg --list-keys' or available 
1097   as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
1098   If not you can put it there and $ME_NAME will autoimport it on the next run.
1099   Alternatively import it manually as the user you plan to run $ME_NAME with."
1100 }
1101 
1102 function error_gpg_test {
1103   [ -n "$2" ] && local hint="\n  $2\n\n  "
1104 
1105   error "$1
1106 
1107 Hint${hint:+s}:
1108   ${hint}This error means that gpg is probably misconfigured or not working 
1109   correctly. The error message above should help to solve the problem.
1110   However, if for some reason $ME_NAME should misinterpret the situation you 
1111   can define GPG_TEST='disabled' in the conf file to bypass the test.
1112   Please do not forget to report the bug in order to resolve the problem
1113   in future versions of $ME_NAME.
1114 "
1115 }
1116 
1117 function error_path {
1118   error "$@
1119 PATH='$PATH'
1120 "
1121 }
1122 
1123 function error_to_string {
1124 	[ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1125 }
1126 
1127 function duplicity_version_get {
1128   # nothing to do, just print
1129   var_isset DUPL_VERSION && return
1130   
1131   local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1132   # only run with a user specific python if configured (running by default
1133   # breaks homebrew as they place a shell wrapper for duplicity in path)
1134   PYTHON_BIN="$(python_binary)" &&\
1135     CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
1136   CMD="$CMD --version 2>&1"
1137   DUPL_VERSION_OUT=$(eval "$CMD")
1138   DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1139   #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1140   DUPL_VERSION_VALUE=0
1141   DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1142   if (match(v,/[^\.0-9]+[0-9]*$/)){
1143     rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1144   if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1145   split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1146   print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1147   print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1148   print "DUPL_VERSION_RC=\047"rc"\047"
1149   print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1150   }')
1151   eval "$DUPL_VERSION_AWK"
1152   #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1153 
1154   # doublecheck findings and report error
1155   if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1156     inform "duplicity version check failed (please report, this is a bug)
1157 the command
1158   $CMD
1159 resulted in
1160   $DUPL_VERSION_OUT
1161 "
1162   elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1163     error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1164 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1165 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1166   fi
1167 }
1168 
1169 function duplicity_version_ge {
1170   [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1171 }
1172 
1173 function duplicity_version_lt {
1174   ! duplicity_version_ge "$1"
1175 }
1176 
1177 # parse interpreter from duplicity shebang
1178 function duplicity_python_binary_parse {
1179   # cached result
1180   ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
1181 
1182   # parse it or warn
1183   local DUPL_BIN=$(which duplicity)
1184   DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1185   if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1186     warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
1187 Will assume it is '$DEFAULT_PYTHON'."
1188     DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
1189   fi
1190 }
1191 
1192 function run_script { # run pre/post scripts
1193   local ERR=0
1194   local SCRIPT="$1"
1195   if [ ! -z "$PREVIEW" ] ; then	
1196     echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1197   elif [ -r "$SCRIPT" ] ; then 
1198     echo -n "Running '$SCRIPT' "
1199     if [ -x "$SCRIPT" ]; then
1200       OUT=$("$SCRIPT" 2>&1)
1201       ERR=$?
1202     else
1203       OUT=$(. "$SCRIPT" 2>&1)
1204       ERR=$?
1205     fi
1206     [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1207     echo -en ${OUT:+"Output: $OUT\n"} ;
1208   else
1209     echo "Skipping n/a script '$SCRIPT'."
1210   fi
1211   return $ERR
1212 }
1213 
1214 function run_cmd {
1215   # run or print escaped cmd string
1216   local CMD_ERR=0
1217   if [ -n "$PREVIEW" ]; then
1218     CMD_OUT=$( echo "$@ 2>&1" )
1219     CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1220   elif [ -n "$CMD_DISABLED" ]; then
1221     CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1222   else
1223     echo -n -e "$CMD_MSG"
1224     CMD_OUT=` eval "$@" 2>&1 `
1225     CMD_ERR=$?
1226     if [ "$CMD_ERR" = "0" ]; then
1227       CMD_MSG=" (OK)"
1228     else
1229       CMD_MSG=" (FAILED)"
1230     fi
1231   fi
1232   echo -e "$CMD_MSG"
1233   # reset
1234   unset CMD_DISABLED CMD_MSG
1235   return $CMD_ERR
1236 }
1237 
1238 function qw { quotewrap "$@"; }
1239 
1240 function quotewrap {
1241   local param="$@"
1242   # quote strings having non word chars (e.g. spaces)
1243   if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1244     echo "$param" | awk '{\
1245       gsub(/[\047]/,"\047\\\047\047",$0);\
1246       gsub(/[\042]/,"\047\\\042\047",$0);\
1247       print "\047"$0"\047"}'
1248     return
1249   fi
1250   echo $param
1251 }
1252 
1253 function duplicity_params_global {
1254   # already done? return
1255   var_isset 'DUPL_PARAMS_GLOBAL' && return
1256   local DUPL_ARG_ENC
1257 
1258   # use key only if set in config, else leave it to symmetric encryption
1259   if gpg_disabled; then
1260     local DUPL_PARAM_ENC='--no-encryption'
1261   else
1262     local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1263     gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1264     # interpret password settings
1265     var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1266     var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1267   fi
1268 
1269   local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1270 
1271   # set name for dupl archive folder, since 0.6.0
1272   if duplicity_version_ge 601; then
1273     local DUPL_ARCHDIR=''
1274     if var_isset 'ARCH_DIR'; then
1275       DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1276       # reuse erronously duply_ prefixed folders from bug #117
1277       if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1278         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1279       else
1280         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1281       fi
1282     else
1283       DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1284     fi
1285   fi
1286 
1287 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1288 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1289  ${GPG_OPTS}"
1290 
1291 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1292  ${DUPL_ARG_ENC}"
1293 }
1294 
1295 # function to filter the DUPL_PARAMS var from user conf
1296 function duplicity_params_conf {
1297   # reuse cmd var from main loop
1298   ## in/exclude parameters are currently not supported on restores
1299   if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1300     # filter exclude params from fetch/restore/status
1301     eval "stripXcludes $DUPL_PARAMS"
1302     return
1303   fi
1304 
1305   # nothing done, print unchanged
1306   echo "$DUPL_PARAMS"
1307 }
1308 
1309 # strip in/exclude parameters from param string
1310 function stripXcludes {
1311   local STRIPNEXT OUT;
1312   for p in "$@"; do
1313     if [ -n "$STRIPNEXT" ]; then
1314       unset STRIPNEXT
1315       # strip the value of previous parameter
1316       continue
1317     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1318       # strips e.g. --include /foo/bar
1319       STRIPNEXT="yes"
1320       continue
1321     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1322       # strips e.g. --include=/foo/bar
1323       continue
1324     fi
1325     
1326     OUT="$OUT $(qw "$p")"
1327   done
1328   echo "$OUT"
1329 }
1330 
1331 function duplify { # the actual wrapper function
1332   local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1333 
1334   # put command (with params) first in duplicity parameters
1335   for param in "$@" ; do
1336   # split cmd from params (everything before splitchar --)
1337     if [ "$param" == "--" ] ; then
1338       PARAMSNOW=1
1339     else
1340       # wrap in quotes to protect from spaces
1341       [ ! $PARAMSNOW ] && \
1342         DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1343       || \
1344         DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1345     fi
1346   done
1347 
1348   # init global duplicity parameters same for all tasks
1349   duplicity_params_global
1350 
1351   local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
1352   # run in cmd line preview mode if requested
1353   var_isset 'PREVIEW' && RUN=echo
1354   # try to resolve duplicity path for usage with python interpreter
1355   DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
1356   # only run with a user specific python if configured (running by default
1357   # breaks homebrew as they place a shell wrapper for duplicity in path)
1358   PYTHON_BIN="$(python_binary)" &&\
1359     BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
1360 
1361 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1362  ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1363  $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1364 
1365   local ERR=$?
1366   return $ERR
1367 }
1368 
1369 function secureconf { # secure the configuration dir
1370 	#PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1371 	local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1372 	if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1373 		chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1374 		warning "The profile's folder 
1375 '$CONFDIR'
1376 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1377 	fi
1378 }
1379 
1380 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1381 function date_fix {
1382 	local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1383 	local date
1384 	#[ "$1" == "%N" ] && return #test the no nsec test below
1385 	# gnu date with -d @epoch
1386 	date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1387 		echo $date && return
1388 	# date bsd,osx with -r epoch
1389 	date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1390 		echo $date && return	
1391 	# date busybox with -d epoch -D %s
1392 	date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1393 		echo $date && return
1394 	## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1395 	# python fallback
1396 	date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1397 		echo $date && return
1398 	# awk fallback
1399 	date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1400 		echo $date && return
1401 	# perl fallback
1402 	date=$(perl  -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1403 		echo $date && return
1404 	# error
1405 	echo "ERROR"
1406 	return 1
1407 }
1408 
1409 function nsecs {
1410 	local NSECS
1411 	# test if date supports nanosecond output
1412 	if ! var_isset NSECS_DISABLED; then
1413 		NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1414 		[ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1415 	fi
1416 
1417 	# add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1418 	if [ "$NSECS_DISABLED" == "1" ]; then
1419 		date_fix %s000000000
1420 	else
1421 		date_fix %s%N
1422 	fi
1423 }
1424 
1425 function nsecs_to_sec {
1426 	echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1427 }
1428 
1429 function datefull_from_nsecs {
1430 	date_from_nsecs $1 '%F %T'
1431 }
1432 
1433 function date_from_nsecs {
1434 	local FORMAT=${2:-%T}
1435 	local TIME=$(nsecs_to_sec $1)
1436 	local SECS=${TIME%.*}
1437 	local DATE=$(date_fix "%T" ${SECS:-0})
1438 	echo $DATE.${TIME#*.}
1439 }
1440 
1441 function var_isset {
1442   if [ -z "$1" ]; then
1443     echo "ERROR: function var_isset needs a string as parameter"
1444   elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1445     return 0
1446   fi
1447   return 1
1448 }
1449 
1450 function is_condition {
1451   local CMD=$(tolower "$1")
1452   [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1453 }
1454 
1455 function is_groupMarker {
1456   local CMD=$(tolower "$1")
1457   [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1458 }
1459 
1460 function is_command {
1461   local CMD=$(tolower "$1")
1462   ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1463 }
1464 
1465 function url_encode {
1466   # utilize python, silently do nothing on error - because no python no duplicity
1467   OUT=$("$(python_binary)" -c "
1468 try: import urllib.request as urllib
1469 except ImportError: import urllib
1470 print(urllib.${2}quote('$1'));
1471 " 2>/dev/null ); ERR=$?
1472   [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1473 }
1474 
1475 function url_decode {
1476   # reuse function above with a simple string param hack
1477   url_encode "$1" "un"
1478 }
1479 
1480 function toupper {
1481   echo "$@"|awk '$0=toupper($0)'
1482 }
1483 
1484 function tolower {
1485   echo "$@"|awk '$0=tolower($0)'
1486 }
1487 
1488 function isnumber {
1489   case "$*" in
1490     ''|*[!0-9]*) return 1;;
1491     *) return 0;;
1492   esac
1493 }
1494 
1495 #function tmp_space {
1496 #  
1497 #  if ! isnumber $VOLSIZE; then
1498 #    inform "failed to determine free space (please report, this is a bug)"
1499 #    return
1500 #  fi
1501 #  
1502 # get free temp space
1503 #  TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1504 #  # check for free space or FAIL
1505 #  if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1506 #    error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1507 #than one duplicity volume (${VOLSIZE}MB).
1508 #    
1509 #  Hint: Free space or change TEMP_DIR setting."
1510 #fi
1511 #
1512 #}
1513 
1514 function gpg_disabled {
1515   echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1516 }
1517 
1518 # usage: join SEPARATOR "entry1" "entry2"
1519 function join {
1520   local SEP="$1" ENTRY OUT; shift;
1521   for ENTRY in "$@"; do
1522     ENTRY=${ENTRY//$SEP/\\$SEP}
1523     [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1524   done
1525   echo $OUT
1526 }
1527 
1528 function gpg_signing {
1529   echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1530 }
1531 
1532 # parameter key id, key_type
1533 function gpg_keyfile {
1534   local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1535   local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1536   echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1537 }
1538 
1539 # parameter key id
1540 function gpg_import {
1541   local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1542   # create a list of legacy key file names and current naming scheme
1543   # we always import pub and sec if they are avail in conf folder
1544   local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1545                    $(gpg_keyfile "$KEY_ID" PUB) $(gpg_keyfile "$KEY_ID" SEC))
1546 
1547   # Try autoimport from existing old gpgkey files 
1548   # and new gpgkey.XXX.asc files (since v1.4.2)
1549   # and even newer gpgkey.XXX.[pub|sec].asc
1550   for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1551     FILE=${KEYFILES[$i]}
1552     if [ -f "$FILE" ]; then
1553       FOUND=1
1554       
1555       CMD_MSG="Import keyfile '$FILE' to keyring"
1556       run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1557       if [ "$?" != "0" ]; then 
1558         warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1559         ERR=1
1560         # continue with next
1561         continue
1562       fi
1563     fi
1564   done
1565 
1566   if [ "$FOUND" -eq 0 ]; then
1567     warning "No keyfile for '$KEY_ID' found in profile\n'$CONFDIR'."
1568   fi
1569 
1570   # try to set trust automagically
1571   CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1572   run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1573   if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then 
1574    # success on all levels, we're done
1575    return $ERR
1576   fi
1577 
1578   # failover: user has to set trust manually
1579   echo -e "For $ME_NAME to work you have to set the trust level 
1580 with the command \"trust\" to \"ultimate\" (5) now.
1581 Exit the edit mode of gpg with \"quit\"."
1582   CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1583   run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1584 
1585   return $ERR
1586 }
1587 
1588 # see 'How to specify a user ID' on gpg manpage
1589 function gpg_fingerprint {
1590   gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1591   awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1592 }
1593 
1594 function gpg_export_if_needed {
1595   local SUCCESS FILE KEY_TYPE
1596   local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1597   for KEY_ID in "$@"; do
1598     # check if already exported, do it if not
1599     for KEY_TYPE in PUB SEC; do
1600       FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1601       if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1602         # exporting
1603         CMD_MSG="Backup $KEY_TYPE key '$KEY_ID' to profile."
1604         # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1605         # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1606         run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1607         CMD_ERR=$?
1608 
1609         if [ "$CMD_ERR" = "0" ]; then
1610           CMD_MSG="Write file '"$(basename "$FILE")"'"
1611           run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1612         fi
1613 
1614         if [ "$CMD_ERR" != "0" ]; then
1615           warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1616         else
1617           SUCCESS=1
1618         fi
1619 
1620         # cleanup
1621         rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1622       fi
1623     done
1624   done
1625   
1626   [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1627 You should backup your changed profile folder now and store it in a safe place."
1628 }
1629 
1630 # replace all non-alnum chars with underscore (for file operations)
1631 function gpg_key_legalize {
1632   echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1633 }
1634 
1635 function gpg_key_cache {
1636   local RES
1637   local MODE=$1
1638   shift
1639   local PREFIX="GPG_KEY"
1640   local SUFFIX=$(gpg_key_legalize "$@")
1641   local KEYID="$*"
1642   local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1643   if [ "$MODE" = "RESET" ]; then
1644     eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1645     return 255
1646   elif ! var_isset "$CACHE"; then
1647     if [ "$MODE" = "PUB" ]; then
1648       RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1649     elif [ "$MODE" = "SEC" ]; then
1650       RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1651     else
1652       return 255
1653     fi
1654     eval $CACHE=$RES
1655   fi
1656   eval return \$$CACHE
1657 }
1658 
1659 function gpg_pub_avail {
1660   gpg_key_cache PUB "$@"
1661 }
1662 
1663 function gpg_sec_avail {
1664   gpg_key_cache SEC "$@"
1665 }
1666 
1667 function gpg_key_format {
1668   echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1669 }
1670 
1671 # splits a comma separated line into lines, respects escaped commas
1672 function gpg_split_keyset {
1673   local LIST
1674   LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1675   echo -e "$LIST"
1676 }
1677 
1678 function gpg_prefix_keyset {
1679   local PREFIX="$1" OUT=""
1680   shift
1681   for KEY_ID in "$@"; do
1682     OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1683   done
1684   echo $OUT
1685 }
1686 
1687 # grep a variable from conf text file (currently not used)
1688 function gpg_passwd {
1689   [ -r "$CONF" ] && \
1690   awk '/^[ \t]*GPG_PW[ \t=]/{\
1691         sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1692         gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1693         print $0; exit}' "$CONF"
1694 }
1695 
1696 # return success if at least one secret key is available
1697 function gpg_key_decryptable {
1698   local KEY_ID
1699   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1700     gpg_sec_avail "$KEY_ID" && return 0
1701   done
1702   return 1
1703 }
1704 
1705 function gpg_symmetric {
1706   [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1707 }
1708 
1709 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1710 function gpg_param_passwd {
1711   var_isset GPG_USEAGENT && exit 1
1712   
1713   if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1714     echo "--passphrase-fd 0 --batch"
1715   fi
1716 }
1717 
1718 # select the earlist defined and create an "echo <value> |" string
1719 function gpg_pass_pipein {
1720   var_isset GPG_USEAGENT && exit 1
1721   
1722   for var in "$@"
1723   do
1724     if var_isset "$var"; then
1725       echo "echo $(qw $(eval echo \$$var)) |"
1726       return 0
1727     fi
1728   done
1729   
1730   return 1
1731 }
1732 
1733 # checks if gpg-agent is available, returns error code
1734 # 0 on success
1735 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1736 # 2 if GPG_AGENT_INFO is stale
1737 # 3 cannot connect to gpg-agent
1738 function gpg_agent_avail {
1739   # GPG_AGENT_INFO is deprecated in gpg2.1, 
1740   # first try to connect to a possibly running agent here
1741   local ERR=3
1742   gpg-agent > /dev/null 2>&1 && return 0
1743 
1744   # detect stale pid in legacy GPG_AGENT_INFO env var
1745   if var_isset GPG_AGENT_INFO; then
1746     # check if a pid matching process is running at all
1747     local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1748     if isnumber "$GPG_AGENT_PID"; then
1749       ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1750     fi
1751   fi
1752 
1753   return $ERR
1754 }
1755 
1756 function gpg_custom_binary {
1757   var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1758     echo "--gpg-binary $(qw "$GPG")"
1759 }
1760 
1761 function gpg_binary {
1762   local BIN
1763   var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1764   echo "$BIN"
1765 }
1766 
1767 function gpg_avail {
1768   lookup "$(gpg_binary)"
1769 }
1770 
1771 # enforce the use of our selected gpg binary
1772 function gpg {
1773   command "$(gpg_binary)" "$@"
1774 }
1775 export -f gpg
1776 
1777 # start of script #######################################################################
1778 
1779 # confidentiality first, all we create is only readable by us
1780 umask 077
1781 
1782 # check if ftplicity is there & executable
1783 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1784 
1785 if [ ${#@} -eq 1 ]; then
1786   cmd="${1}"
1787 else
1788   FTPLCFG="${1}" ; cmd="${2}"
1789 fi
1790 
1791 # deal with command before profile validation calls
1792 # show requested version
1793 # OR requested usage info
1794 # OR create a profile
1795 # OR fall through
1796 ##if [ ${#@} -le 2 ]; then
1797 case "$cmd" in
1798   changelog)
1799     changelog
1800     exit 0
1801     ;;
1802   create)
1803     set_config
1804     if [ -d "$CONFDIR" ]; then
1805       error "The profile '$FTPLCFG' already exists in
1806 '$CONFDIR'.
1807 
1808 Hint:
1809  If you _really_ want to create a new profile by this name you will 
1810  have to manually delete the existing profile folder first."
1811       exit 1
1812     else
1813       create_config
1814       exit 0
1815     fi
1816     ;;
1817   txt2man)
1818     set_config
1819     usage_txt2man
1820     exit 0
1821     ;;
1822   usage|-help|--help|-h|-H)
1823     set_config
1824     usage_info
1825     exit 0
1826     ;;
1827   version|-version|--version|-v|-V)
1828     # profile can override GPG/PYTHON, so import it if it was given
1829     var_isset FTPLCFG && {
1830       set_config
1831       [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1832     }
1833     version_info_using
1834     exit 0
1835     ;;
1836   # fallthrough.. we got a command that needs an existing profile
1837   *)
1838     # if we reach here, user either forgot profile or chose wrong profileless command
1839     if [ ${#@} -le 1 ]; then
1840       error "\
1841  Missing or wrong parameters. 
1842  Only the commands 
1843    changelog, create, usage, txt2man, version
1844  can be called without selecting an existing profile first.
1845  Your command was '$cmd'.
1846 
1847  Hint: Run '$ME usage' to get help."
1848     fi
1849 esac
1850 
1851 
1852 # Hello world
1853 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1854 
1855 # check system environment
1856 
1857 # is duplicity avail
1858 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1859 # init, exec duplicity version check info
1860 duplicity_python_binary_parse
1861 duplicity_version_get
1862 
1863 # check for certain important helper programs
1864 for f in awk grep "$(python_binary)"; do
1865   lookup "$f" || \
1866     error_path "$f missing. installed und available in path?"
1867 done
1868 
1869 ### read configuration
1870 set_config
1871 # check validity
1872 if [ ! -d "$CONFDIR" ]; then 
1873     error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1874 '$CONFDIR'.
1875 
1876 Hints:
1877 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1878  Use '$ME <name> create' to create a new profile.
1879  Use '$ME usage' to get usage help."
1880 elif [ ! -x "$CONFDIR" ]; then
1881     error "\
1882 Profile folder in '$CONFDIR' cannot be accessed.
1883 
1884 Hint: 
1885  Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1886 elif [ ! -f "$CONF" ] ; then
1887   error "'$CONF' not found."
1888 elif [ ! -r "$CONF" ] ; then
1889   error "'$CONF' not readable."
1890 else
1891   . "$CONF"
1892   #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1893   TEMP_DIR=${TEMP_DIR:-'/tmp'}
1894   # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1895   if var_isset 'TARGET_PW'; then
1896     TARGET_PASS="${TARGET_PW}"
1897   fi
1898 fi
1899 echo "Using profile '$CONFDIR'."
1900 
1901 # secure config dir, if needed w/ warning
1902 secureconf
1903 
1904 # split TARGET in handy variables
1905 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1906   target=$0; match(target,/^([^\/:]+):\/\//); \
1907   prot=substr(target,RSTART,RLENGTH);\
1908   rest=substr(target,RSTART+RLENGTH); \
1909   if (credsavail=match(rest,/^[^@]*@/)){\
1910     creds=substr(rest,RSTART,RLENGTH-1);\
1911     credcount=split(creds,cred,":");\
1912     rest=substr(rest,RLENGTH+1);\
1913     # split creds with regexp\
1914     match(creds,/^([^:]+)/);\
1915     user=substr(creds,RSTART,RLENGTH);\
1916     pass=substr(creds,RSTART+1+RLENGTH);\
1917   };\
1918   # filter quotes or escape them\
1919   gsub(/[\047\042]/,"",prot);\
1920   gsub(/[\047\042]/,"",rest);\
1921   gsub(/[\047]/,"\047\\\047\047",creds);\
1922   print "TARGET_URL_PROT=\047"prot"\047\n"\
1923          "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1924          "TARGET_URL_CREDS=\047"creds"\047\n";\
1925    if(user){\
1926      gsub(/[\047]/,"\047\\\047\047",user);\
1927      print "TARGET_URL_USER=\047"user"\047\n"}\
1928    if(pass){\
1929      gsub(/[\047]/,"\047\\\047\047",pass);\
1930      print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1931   }')
1932 eval "${TARGET_SPLIT_URL}"
1933 
1934 # fetch commmand from parameters ########################################################
1935 # Hint: cmds is also used to check if authentification info sufficient in the next step 
1936 cmds="$2"; shift 2
1937 
1938 # complain if command(s) missing
1939 [ -z $cmds ] && error "  No command given.
1940 
1941   Hint: 
1942     Use '$ME usage' to get usage help."
1943 
1944 # process params
1945 for param in "$@"; do
1946   #echo !$param!
1947   case "$param" in
1948     # enable ftplicity preview mode
1949     '--preview')
1950       PREVIEW=1
1951       ;;
1952     # interpret duplicity disable encr switch
1953     '--disable-encryption')
1954       GPG_KEY='disabled'
1955       ;;
1956     *)
1957       if [ `echo "$param" | grep -e "^-"` ] || \
1958          [ `echo "$last_param" | grep -e "^-"` ] ; then
1959         # forward parameter[/option pairs] to duplicity
1960         dupl_opts["${#dupl_opts[@]}"]=${param}
1961       else
1962         # anything else must be a parameter (e.g. for fetch, ...)
1963         ftpl_pars["${#ftpl_pars[@]}"]=${param}
1964       fi
1965       last_param=${param}
1966       ;;
1967   esac
1968 done
1969 
1970 # plausibility check config - VARS & KEY ################################################
1971 # check if src, trg, trg pw
1972 # auth info sufficient 
1973 # gpg key, gpg pwd (might be empty) set in config
1974 # OR key in local gpg db
1975 # OR key can be imported from keyfile 
1976 # OR fail
1977 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
1978  error " Source Path (setting SOURCE) not set or still default value in conf file 
1979  '$CONF'."
1980 
1981 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
1982  error " Backup Target (setting TARGET) not set or still default value in conf file 
1983  '$CONF'."
1984 
1985 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
1986      [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
1987  error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}') 
1988  are configured with different values. There can be only one.
1989  
1990  Hint: Remove conflicting setting."
1991 
1992 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
1993      [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
1994  error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}') 
1995  are configured with different values. There can be only one.
1996  
1997  Hint: Remove conflicting setting."
1998 fi
1999 
2000 # GPG config plausibility check1 (disabled check) #############################
2001 if gpg_disabled; then
2002   : # encryption disabled, all is well
2003 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2004   warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file 
2005 '$CONF'.
2006 Will disable encryption for duplicity now.
2007 
2008 Hint: 
2009  If you really want to use _no_ encryption you can disable this warning by 
2010  setting GPG_KEY='disabled' in conf file."
2011   GPG_KEY='disabled'
2012 fi
2013 
2014 # GPG availability check (now we know if gpg is really needed)#################
2015 if ! gpg_disabled; then 
2016   gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2017 fi
2018 
2019 # Output versions info ########################################################
2020 using_info
2021 
2022 # GPG create key settings, config check2 (needs gpg) ##########################
2023 if gpg_disabled; then
2024   : # the following tests are not necessary
2025 else
2026 
2027 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2028 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2029 
2030 # enc key still default?
2031 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then 
2032   error_gpg "Encryption Key GPG_KEY still default in conf file 
2033 '$CONF'."
2034 fi
2035 
2036 # create array of gpg encr keys, for further processing
2037 OIFS="$IFS" IFS=$'\n'
2038 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2039 IFS="$OIFS"
2040 
2041 # pw set? 
2042 # symmetric needs one, always
2043 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2044   ; then
2045   error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) 
2046 is empty/not set or still default value in conf file 
2047 '$CONF'."
2048 fi
2049 # this is a technicality, we can only pump one pass via pipe into gpg
2050 # but symmetric already always needs one for encryption
2051 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2052   [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2053   error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2054 '$CONF'.
2055 This is unfortunately impossible. For details see duplicity manpage, 
2056 section 'A Note On Symmetric Encryption And Signing'.
2057 
2058 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2059 Tip2: Use gpg-agent."
2060 fi
2061 
2062 # check gpg encr public keys availability
2063 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2064   KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2065   # test availability, try to import, retest
2066   if ! gpg_pub_avail "${KEY_ID}"; then
2067     echo "Encryption public key '${KEY_ID}' not found."
2068     gpg_import "${KEY_ID}" PUB
2069     gpg_key_cache RESET "${KEY_ID}"
2070     gpg_pub_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Public"
2071   fi
2072 done
2073 
2074 # gpg secret sign key availability
2075 # if none set, autoset first encryption key as sign key
2076 if ! gpg_signing; then
2077   echo "Signing disabled per configuration."
2078 # try first key, if one set
2079 elif ! var_isset 'GPG_KEY_SIGN'; then
2080   KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2081   if [ -z "${KEY_ID}" ]; then
2082     echo "Signing disabled. No GPG_KEY entries in config."
2083     GPG_KEY_SIGN='disabled'
2084   else  
2085     # use avail OR try import OR fail
2086     if gpg_sec_avail "${KEY_ID}"; then
2087       GPG_KEY_SIGN="${KEY_ID}"
2088     else
2089       gpg_import "${KEY_ID}" SEC
2090       gpg_key_cache RESET "${KEY_ID}"
2091       if gpg_sec_avail "${KEY_ID}"; then
2092         GPG_KEY_SIGN="${KEY_ID}"
2093       fi
2094     fi
2095 
2096     # interpret sign key setting
2097     if var_isset 'GPG_KEY_SIGN'; then
2098       echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2099     else
2100       echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2101       GPG_KEY_SIGN='disabled'
2102     fi
2103   fi
2104 else
2105   KEY_ID="${GPG_KEY_SIGN}"
2106   if ! gpg_sec_avail "${KEY_ID}"; then
2107     inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2108     gpg_import "${KEY_ID}" SEC
2109     gpg_key_cache RESET "${KEY_ID}"
2110     gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2111   fi
2112 fi
2113 
2114 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2115 # key enc can deal without, but might profit from gpg-agent
2116 # if GPG_PW is not set alltogether
2117 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2118 if ! gpg_symmetric && \
2119    ( ! var_isset GPG_PW || \
2120      ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2121 
2122   if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2123     warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2124   elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2125     warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2126   elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2127     warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2128   else
2129     echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2130     GPG_USEAGENT="--use-agent"
2131   fi
2132 fi
2133 
2134 # end GPG config plausibility check2 
2135 fi
2136 
2137 # config plausibility check - SPACE ###########################################
2138 
2139 # is tmp is a folder and writable
2140 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2141 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2142 if [ "$?" != "0" ]; then
2143     error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2144 fi
2145 
2146 
2147 # get volsize, default duplicity volume size is 25MB since v0.5.07
2148 VOLSIZE=${VOLSIZE:-25}
2149 # double if asynch is on
2150 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2151 
2152 # TODO: check for enough (async= upload space and WARN only
2153 #       use function tmp_space 
2154 #echo TODO: reimplent tmp space check
2155 
2156 
2157 # test - GPG SANITY #####################################################################
2158 # if encryption is disabled, skip this whole section
2159 if gpg_disabled; then
2160   echo -e "Test - En/Decryption skipped. (GPG disabled)"
2161 elif [ "$GPG_TEST" = "disabled" ]; then 
2162   echo -e "Test - En/Decryption skipped. (Testing disabled)"
2163 else
2164 
2165 GPG_TEST="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2166 function cleanup_gpgtest { 
2167   echo -en "Cleanup - Delete '${GPG_TEST}_*'"
2168   rm "${GPG_TEST}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2169 }
2170 
2171 # signing enabled?
2172 if gpg_signing; then
2173   CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2174   CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2175 fi
2176 
2177 # using keys
2178 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2179 
2180   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2181     CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2182   done
2183   # check encrypting
2184   CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2185   run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST}_ENC") -e $(qw "$ME_LONG")
2186   CMD_ERR=$?
2187 
2188   if [ "$CMD_ERR" != "0" ]; then 
2189     KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2190     [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2191   'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2192   can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2193     error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2194   fi
2195 
2196   # check decrypting
2197   CMD_MSG="Test - Decrypt"
2198   gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2199   run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST}_ENC")
2200   CMD_ERR=$?
2201 
2202   if [ "$CMD_ERR" != "0" ]; then 
2203     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2204   fi
2205 
2206 # symmetric only
2207 else
2208   # check encrypting
2209   CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2210   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST}_ENC") --batch -c $(qw "$ME_LONG")
2211   CMD_ERR=$?
2212   if [ "$CMD_ERR" != "0" ]; then 
2213     error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2214   fi
2215 
2216   # check decrypting
2217   CMD_MSG="Test - Decryption with passphrase"
2218   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST}_DEC") --batch -d $(qw "${GPG_TEST}_ENC")
2219   CMD_ERR=$?
2220   if [ "$CMD_ERR" != "0" ]; then 
2221     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2222   fi
2223 fi
2224 
2225 # compare original w/ decryptginal
2226 CMD_MSG="Test - Compare"
2227 [ -r "${GPG_TEST}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2228 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST}_DEC')\""
2229 CMD_ERR=$?
2230 if [ "$CMD_ERR" = "0" ]; then 
2231   cleanup_gpgtest
2232 else
2233   error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2234 fi
2235 
2236 fi # end disabled
2237 
2238 ## an empty line
2239 #echo
2240 
2241 # Exclude file is needed, create it if necessary
2242 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2243 
2244 # export only used keys, if bkp not already exists ######################################
2245 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2246 
2247 
2248 # command execution #####################################################################
2249 
2250 # urldecode url vars into plain text
2251 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2252 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2253 
2254 # defined TARGET_USER&PASS vars replace their URL pendants 
2255 # (double defs already dealt with)
2256 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2257 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2258 
2259 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2260 
2261 # issue some warnings
2262 case "$TARGET_URL_PROT_lowercase" in
2263   'cf+http')
2264     # info on missing AUTH_URL
2265     if ! var_isset 'CLOUDFILES_AUTHURL'; then
2266       inform "No CLOUDFILES_AUTHURL exported (in conf).
2267 Will use default which is probably rackspace."
2268     fi
2269     ;;
2270    'swift')
2271     # info on possibly missing AUTH_URL
2272        var_isset 'SWIFT_AUTHURL' ||\
2273        warning "\
2274 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2275     ;;
2276   'rsync')
2277     # everything in url (this backend does not support pass in env var)
2278     # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2279     # print warning older version is detected
2280     duplicity_version_lt 610 &&
2281       warning "\
2282 Duplicity version '$DUPL_VERSION' does not support providing the password as 
2283 env var for rsync backend. For security reasons you should consider to 
2284 update to a version greater than '0.6.10' of duplicity."
2285     ;;
2286 esac
2287 
2288 
2289 # for all protocols we put username in url and pass into env var 
2290 # for sec�rity reasons, we url_encode username to protect special chars
2291 # first sortout backends with special ways to handle password
2292 case "$TARGET_URL_PROT_lowercase" in
2293   'imap'|'imaps')
2294     var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2295     ;;
2296   *)
2297     # rest uses FTP_PASS var
2298     var_isset 'TARGET_URL_PASS' && \
2299       BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2300     ;;
2301 esac
2302 # insert url encoded username into target url if needed
2303 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2304   BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2305 else
2306   BACKEND_URL="$TARGET"
2307 fi
2308 
2309 
2310 # protect eval from special chars in url (e.g. open ')' in password, 
2311 # spaces in path, quotes) happens above in duplify() via quotewrap()
2312 SOURCE="$SOURCE"
2313 BACKEND_URL="$BACKEND_URL"
2314 EXCLUDE="$EXCLUDE"
2315 # since 0.7.03 --exclude-globbing-filelist is deprecated
2316 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist" 
2317 
2318 # translate backup to batch command 
2319 cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
2320 
2321 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2322 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2323   gsub(/\+/,\"_and_\",cmds);\
2324   gsub(/\-/,\"_or_\",cmds);\
2325   gsub(/\[/,\"_groupIn_\",cmds);\
2326   gsub(/\]/,\"_groupOut_\",cmds);\
2327   print cmds}")
2328 # convert cmds to array, lowercase for safety
2329 declare -a CMDS
2330 CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
2331 
2332 unset FTPL_ERR
2333 
2334 # run cmds
2335 for cmd in ${CMDS[*]};
2336 do
2337 
2338 ## init
2339 # raise index in cmd array for pre/post param
2340 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2341 
2342 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2343 
2344 # get next cmd,cnd vars
2345 nextno=$(( $CMD_NO ))
2346 while ! var_isset 'CMD_NEXT'
2347 do
2348   nextno=$(($nextno+1))
2349   if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2350     CMD_VALUE=${CMDS[$nextno]}
2351     is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2352     is_groupMarker "$CMD_VALUE" && continue
2353     CMD_NEXT="$CMD_VALUE"
2354   else
2355     CMD_NEXT='END'
2356   fi
2357 done
2358 
2359 # get prev cnd, cnds are skipped pseudocmds
2360 prevno=$(( $CMD_NO ));
2361 while ! var_isset 'CND_PREV'
2362 do
2363   prevno=$(($prevno-1))
2364   if [ "$prevno" -ge 0 ]; then
2365     CMD_VALUE=${CMDS[$prevno]}
2366     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2367     is_command "$CMD_VALUE" && break
2368   else
2369     break
2370   fi
2371 done
2372 
2373 # get prev cmd command minus skipped commands, only executed
2374 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2375 while ! var_isset 'CMD_PREV'
2376 do
2377   prevno=$(($prevno-1))
2378   if [ "$prevno" -ge 0 ]; then
2379     CMD_VALUE=${CMDS[$prevno]}
2380     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2381     is_groupMarker "$CMD_VALUE" && continue
2382     CMD_PREV="$CMD_VALUE"
2383   else
2384     CMD_PREV='START'
2385   fi
2386 done
2387 
2388 function get_cmd_skip_count {
2389   # find closing bracket, get group skip count
2390   local nextno=$CMD_NO
2391   local GRP_OPEN=0
2392   local GRP_SKIP=0
2393   local CMD_VALUE
2394   while [ "$nextno" -lt "${#CMDS[@]}" ]
2395   do
2396     nextno=$(($nextno+1))
2397     CMD_VALUE=${CMDS[$nextno]}
2398     GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2399     if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2400         break;
2401     elif [ "$CMD_VALUE" == 'groupin' ]; then
2402       GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2403     elif [ "$CMD_VALUE" == 'groupout' ]; then
2404       GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2405       if [ "$GRP_OPEN" -lt 1 ]; then
2406         break;
2407       fi
2408     fi
2409   done
2410 
2411   echo $GRP_SKIP;
2412 }
2413 
2414 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2415 unset SKIP_NOW
2416 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2417   # skip cnd/grp cmds silently
2418   is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2419   CMD_SKIP=$(($CMD_SKIP - 1))
2420   SKIP_NOW="yes"
2421 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2422   CMD_SKIP=$(get_cmd_skip_count)
2423   # incl. this "cmd"
2424   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2425   unset CMD_SKIPPED
2426   SKIP_NOW="yes"
2427 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2428   CMD_SKIP=$(get_cmd_skip_count)
2429   # incl. this "cmd"
2430   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2431   unset CMD_SKIPPED
2432   SKIP_NOW="yes"
2433 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2434   unset 'CMD_SKIP';
2435   SKIP_NOW="yes"
2436 fi
2437 
2438 # let's do the skip now
2439 if [ -n "$SKIP_NOW" ]; then
2440   # sum up how many commands we actually skipped for the prev var routines
2441   CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2442   continue
2443 fi
2444 
2445 # save start time
2446 RUN_START=$(nsecs)
2447 
2448 # export some useful env vars for external scripts/programs to use
2449 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2450        TARGET_URL_USER TARGET_URL_PASS \
2451        GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2452        GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2453        RUN_START
2454 
2455 # user info
2456 echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
2457 
2458 case "$(tolower $cmd)" in
2459   'pre'|'post')
2460     if [ "$cmd" == 'pre' ]; then
2461       script=$PRE
2462     else
2463       script=$POST
2464     fi
2465     # script execution in a subshell, protect us from failures/var overwrites
2466     ( run_script "$script" )
2467     ;;
2468   'bkp')
2469     duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2470           "$SOURCE" "$BACKEND_URL"
2471     ;;
2472   'incr')
2473     duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2474           "$SOURCE" "$BACKEND_URL"
2475     ;;
2476   'full')
2477     duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2478           "$SOURCE" "$BACKEND_URL"
2479     ;;
2480   'verify')
2481     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2482     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2483           "$BACKEND_URL" "$SOURCE"
2484     ;;
2485   'verifypath')
2486     TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2487     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2488     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2489   
2490   Hint: 
2491     Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2492 
2493     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2494           --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2495     ;;
2496   'list')
2497     # time param exists since 0.5.10+
2498     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2499     duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2500     ;;
2501   'cleanup')
2502     duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2503     ;;
2504   'purge')
2505     MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2506     [ -z "$MAX_AGE" ] && error "  Missing parameter <max_age>. Can be set in profile or as command line parameter."
2507     
2508     duplify remove-older-than "${MAX_AGE}" \
2509           -- "${dupl_opts[@]}" "$BACKEND_URL"
2510     ;;
2511   'purgefull')
2512     MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2513     [ -z "$MAX_FULL_BACKUPS" ] && error "  Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2514   
2515     duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2516           -- "${dupl_opts[@]}" "$BACKEND_URL"
2517     ;;
2518   'purgeincr')
2519     MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2520     [ -z "$MAX_FULLS_WITH_INCRS" ] && error "  Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2521   
2522     duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2523           -- "${dupl_opts[@]}" "$BACKEND_URL"
2524     ;;
2525   'restore')
2526     OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2527     [ -z "$OUT_PATH" ] && error "  Missing parameter target_path for restore.
2528   
2529   Hint: 
2530     Syntax is -> $ME <profile> restore <target_path> [<age>]"
2531     
2532     duplify  -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2533     ;;
2534   'fetch')
2535     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2536     TIME="${ftpl_pars[2]:-now}";
2537     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <src_path> or <target_path> for fetch.
2538   
2539   Hint: 
2540     Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2541     
2542     # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2543     duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2544               --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2545     ;;
2546   'status')
2547     duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2548     ;;    
2549   *)
2550     error "  Unknown command '$cmd'.
2551 
2552   Hint:
2553     Use '$ME usage' to get usage help."
2554     ;;
2555 esac
2556 
2557 CMD_ERR=$?
2558 RUN_END=$(nsecs)
2559 RUNTIME=$(( $RUN_END - $RUN_START ))
2560 
2561 # print message on error; set error code
2562 if [ "$CMD_ERR" -ne 0 ]; then
2563 	error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2564 	FTPL_ERR=1
2565 fi
2566 
2567 separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
2568 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2569 
2570 done
2571 
2572 exit ${FTPL_ERR}