Sourcecode / SVN

From duply (simple duplicity) - a duplicity shell frontend
Jump to navigation Jump to search


duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/

SVN access

svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply

SVN Log

https://sourceforge.net/p/ftplicity/code/commit_browser

Latest Development Snapshot

mod time 2022-04-06

plain/text -> duply.sh

   1 #!/usr/bin/env bash
   2 #
   3 ################################################################################
   4 #  duply (grown out of ftplicity), is a shell front end to duplicity that      #
   5 #  simplifies the usage by managing settings for backup jobs in profiles.      #
   6 #  It supports executing multiple commands in a batch mode to enable single    #
   7 #  line cron entries and executes pre/post backup scripts.                     #
   8 #  Since version 1.5.0 all duplicity backends are supported. Hence the name    #
   9 #  changed from ftplicity to duply.                                            #
  10 #  See http://duply.net or http://ftplicity.sourceforge.net/ for more info.    #
  11 #  (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany            #
  12 #  (c) 2008-2022 Edgar Soldin (changes since version 1.3)                      #
  13 ################################################################################
  14 #  LICENSE:                                                                    #
  15 #  This program is licensed under GPLv2.                                       #
  16 #  Please read the accompanying license information in gpl-2.0.txt.            #
  17 ################################################################################
  18 #  TODO/IDEAS/KNOWN PROBLEMS:
  19 #  - possibility to restore time frames (incl. deleted files)
  20 #    realizable by listing each backup and restore from 
  21 #    oldest to the newest, problem: not performant
  22 #  - search file in all backups function and show available
  23 #    versions with backups date (list old avail since 0.6.06)
  24 #  - edit profile opens conf file in vi 
  25 #  - implement log-fd interpretation
  26 #  - add a duplicity option check against the options pending 
  27 #    deprecation since 0.5.10 namely --time-separator
  28 #                               --short-filenames
  29 #                              --old-filenames
  30 #  - add 'exclude_<command>' list usage e.g. exclude_verify
  31 #  - featreq 25: a download/install duplicity option
  32 #  - import/export profile from/to .tgz function !!!
  33 #  - remove url_encode, test for invalid chars n throw error instead
  34 #
  35 #  CHANGELOG:
  36 #  2.4 (6.4.2022)
  37 #  - bugfix #127: date_from_nsecs ignores format string
  38 #  - bugfix #116: separators print date now too
  39 #  - featreq #48: add purgeAuto command (see man page)
  40 #  - replaced tab indents with 2spaces everywhere
  41 #  - bugfix #129,131,132: duply stumbles over 'python -s' shebang,
  42 #      python interpreter parse failed if duplicity is a snap app
  43 #  - bugfix #130: duplicity version check failed "gpg: WARNING: ..."
  44 #  - version output, always print PYTHONPATH, if interpreter was determined
  45 #  - update python references to python3
  46 #
  47 #  2.3.1 (11.2.2021)
  48 #  - bugfix 123: symmetric encryption errs out, asks for '' private key
  49 #
  50 #  2.3 (30.12.2020)
  51 #  - don't import whole key pair anymore if only pub/sec is requested
  52 #  - gpg import routine informs on missing key files in profile now
  53 #  - add check/import needed secret key for decryption
  54 #  - featreq 50: Disable GPG key backups, implemented/added settings
  55 #      GPG_IMPORT/GPG_EXPORT='disabled' to conf template
  56 #
  57 #  2.2.2 (24.02.2020)
  58 #  - bugfix 120: Failures in "Autoset trust of key" during restore 
  59 #    because of gpg2.2 fingerprint output change
  60 #
  61 #  2.2.1 (22.01.2020)
  62 #  - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  63 #  - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  64 #  - fix abbreviation spelling of 'e.g.'
  65 #
  66 #  2.2 (30.12.2018)
  67 #  - featreq 44: implement grouping for batch commands
  68 #      new separators are [] (square brackets) or groupIn/groupOut
  69 #      command 'backup' translates now to [pre_bkp_post] to be skipped as
  70 #      one block in case a condition was set in the batch instruction
  71 #
  72 #  2.1 (23.07.2018)
  73 #  - be more verbose when duplicity version detection fails
  74 #  - using info shows python binary's path for easier identification now
  75 #  - reworked python interpreter handling, it's either
  76 #      configured per PYTHON var
  77 #      unconfigured, parsed from duplicity shebang
  78 #      or set to current duplicity default 'python2' (was 'python' until now)
  79 #  - do not quotewrap strings because of slashes (e.g. paths) anymore
  80 #  - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  81 #
  82 #  2.0.4 (20.02.2018)
  83 #  - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  84 #  - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  85 #  - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  86 #  - bugfix debian 882159: duply: occasionally shows negative runtimes
  87 #
  88 #  2.0.3 (29.08.2017)
  89 #  - bugfix: "line 2231: CMDS: bad array subscript"
  90 #  - bugfix 112: "env: illegal option -- u" on MacOSX
  91 #
  92 #  2.0.2 (23.05.2017)
  93 #  - bugfix: never insert creds into file:// targets
  94 #  - bugfix: avail profiles hint sometimes shortend the names by one char
  95 #  - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  96 #  - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  97 #  - bugfix 97: Unknown command should be ERROR, not WARNING 
  98 #
  99 #  2.0.1 (16.11.2016)
 100 #  - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
 101 #
 102 #  2.0 (27.10.2016)
 103 #  made this a major version change, as we broke backward compatibility anyway
 104 #  (see last change in v1.10). got complaints that rightfully pointed out
 105 #  that should only come w/ a major version change. so, here we go ;)
 106 #  if your backend stops working w/ this version create a new profile and
 107 #  export the env vars needed as described in the comments of the conf file
 108 #  directly above the SOURCE setting.
 109 #  Changes:
 110 #  - making sure multi spaces in TARGET survive awk processing
 111 #  - new env var PROFILE exported to scripts 
 112 #  - fix 102: expose a unique timestamp variable for pre/post scripts
 113 #    actually a featreq. exporting RUN_START nanosec unix timestamp
 114 #  - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
 115 #  - fix 96: duply cannot handle two consecutive spaces in paths
 116 #
 117 #  1.11.3 (29.5.2016)
 118 #  - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
 119 #
 120 #  1.11.2 (11.2.2016)
 121 #  - fix "gpg: unsafe" version print out 
 122 #  - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
 123 #  - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional 
 124 #    documentation about needed env vars to template conf file
 125 #
 126 #  1.11.1 (18.12.2015)
 127 #  - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
 128 #  - reverted duply's default PYTHON to 'python'
 129 #
 130 #  1.11 (24.11.2015)
 131 #  - remove obsolete --ssh-askpass routine
 132 #  - add PYTHON conf var to allow global override of used python interpreter
 133 #  - enforced usage of "python2" in PATH as default interpreter for internal
 134 #    use _and_ to run duplicity (setup.py changed the shebang to the fixed
 135 #    path /usr/bin/python until 0.7.05, which we circumvent this way)
 136 #  - featreq 36: support gpg-connect-agent as a means to detect if an agent is 
 137 #    running (thx Thomas Harning Jr.), used gpg-agent for detection though
 138 #  - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
 139 #  - key export routine respects gpg-agent usage now
 140 #
 141 #  1.10.1 (19.8.2015)
 142 #  - bugfix 86: Duply+Swift outputs warning
 143 #  - bugfix 87: Swift fails without BACKEND_URL
 144 #
 145 #  1.10 (31.7.2015)
 146 #  - featreq 37: busybox issues - fix awk, grep version detection,
 147 #    fix grep failure because --color=never switch is unsupported
 148 #    (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
 149 #  - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
 150 #    (thx Joachim Wiedorn, also for maintaining the debian package)
 151 #  - implemented base-/dirname as bash functions
 152 #  - featreq 31 " Support for duplicity Azure backend " - ignored a 
 153 #    contributed patch by Scott McKenzie and instead opted for removing almost
 154 #    all code that deals with special env vars required by backends.
 155 #    adding and modifying these results in too much overhead so i dropped this
 156 #    feature. the future alternative for users is to consult the duplicity 
 157 #    manpage and add the needed export definitions to the conf file.
 158 #    appended a commented example to the template conf below the auth section.
 159 #
 160 #  1.9.2 (21.6.2015)
 161 #  - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
 162 #  - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
 163 #  - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
 164 #  - added GPG conf var (see conf template for details)
 165 #  - added grep version output as it is an integral needed binary
 166 #  - added PYTHONPATH printout in version output
 167 #
 168 #  1.9.1 (13.10.2014)
 169 #  - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
 170 #  - bugfix: CMD_PREV contained command even if it was skipped
 171 #
 172 #  1.9.0 (24.8.2014)
 173 #  - bugfix: env vars were not exported when external script was executable
 174 #  - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) 
 175 #    see gpg manpage, section "How to specify a user ID"
 176 #    let gpg complain when the delivered values are invalid for whatever reason
 177 #  - started to rework tmp space checking, exposed folder & writable check
 178 #    TODO: reimplement enough file space available checking
 179 #
 180 #  1.8.0 (13.7.2014)
 181 #  - add command verifyPath to expose 'verify --file-to-restore' action
 182 #  - add time parameter support to verify command
 183 #  - add section time formats to usage output 
 184 #
 185 #  1.7.4 (24.6.2014)
 186 #  - remove ubuntu one support, service is discontinued
 187 #  - featreq 31: add authenticated swift (contributed by Justus Seifert)
 188 #
 189 #  1.7.3 (3.4.2014)
 190 #  - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
 191 #
 192 #  1.7.2 (1.4.2014 "April,April")
 193 #  - bugfix: debian Bug#743190 "duply no longer allows restoration without 
 194 #     gpg passphrase in conf file"
 195 #     GPG_AGENT_INFO env var is now needed to trigger --use-agent
 196 #  - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
 197 #
 198 #  1.7.1 (30.3.2014)
 199 #  - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to 
 200 #     incompatibility with new minus batch separator 
 201 #
 202 #  1.7.0 (20.3.2014)
 203 #  - disabled gpg key id plausibility check, too many valid possibilities
 204 #  - featreq 7 "Halt if precondition fails":
 205 #     added and(+), or(-) batch command(separator) support
 206 #  - featreq 26 "pre/post script with shebang line": 
 207 #     if a script is flagged executable it's executed in a subshell 
 208 #     now as opposed to sourced to bash, which is the default
 209 #  - bugfix: do not check if dpbx, swift credentials are set anymore 
 210 #  - bugfix: properly escape profile name, archdir if used as arguments
 211 #  - add DUPL_PRECMD conf setting for use with e.g. trickle
 212 #
 213 #  1.6.0 (1.1.2014)
 214 #  - support gs backend
 215 #  - support dropbox backend
 216 #  - add gpg-agent support to gpg test routines
 217 #  - autoenable --use-agent if passwords were not defined in config
 218 #  - GPG_OPTS are now honored everywhere, keyrings or complete gpg
 219 #    homedir can thus be configured to be located anywhere
 220 #  - always import both secret and public key if avail from config profile
 221 #  - new explanatory comments in initial exclude file
 222 #  - bugfix 7: Duply only imports one key at a time 
 223 #
 224 #  1.5.11 (19.07.2013)
 225 #  - purge-incr command for remove-all-inc-of-but-n-full feature added
 226 #    patch provided by Moritz Augsburger, thanks!
 227 #  - documented version command in man page
 228 #
 229 #  1.5.10 (26.03.2013)
 230 #  - minor indent and documentation fixes
 231 #  - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
 232 #  - bugfix: fix url_decoding generally and for python3
 233 #  - bugfix 3609075: wrong script results in status line (thx David Epping)
 234 #
 235 #  1.5.9 (22.11.2012)
 236 #  - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
 237 #  - restore/fetch now also ignores --include* or --exclude='foobar' 
 238 #
 239 #  1.5.8 (26.10.2012)
 240 #  - bugfix 3575487: implement proper cloud files support
 241 #
 242 #  1.5.7 (10.06.2012)
 243 #  - bugfix 3531450: Cannot use space in target URL (file:///) anymore
 244 #
 245 #  1.5.6 (24.5.2012)
 246 #  - commands purge, purge-full have no default value anymore for security 
 247 #    reasons; instead max value can be given via cmd line or must be set
 248 #    in profile; else an error is shown.
 249 #  - minor man page modifications
 250 #
 251 #  versioning scheme will be simplified to [major].[minor].[patch] version
 252 #  with the next version raise
 253 #
 254 #  1.5.5.5 (4.2.2012)
 255 #  - bugfix 3479605: SEL context confused profile folder's permission check
 256 #  - colon ':' in url passphrase got ignored, added python driven url_decoding
 257 #    for user & pass to better process special chars
 258 #
 259 #  1.5.5.4 (16.10.2011)
 260 #  - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
 261 #  - add support for separate sign passphrase (needs duplicity 0.6.14+)
 262 #
 263 #  1.5.5.3 (1.10.2011)
 264 #  - bugfix 3416690: preview threw echo1 error
 265 #  - fix unknown cmds error usage & friends if more than 2 params were given
 266 #
 267 #  1.5.5.2 (23.9.2011)
 268 #  - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
 269 #  - bugfix: mawk does not support \W and did not split multikey definitions
 270 #  - all parameters should survive  single (') and double (") quotes now
 271 #
 272 #  1.5.5.1 (7.6.2011)
 273 #  - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
 274 #  - bugfix 3312208: signing detection broke symmetric gpg test routine
 275 #
 276 #  1.5.5 (2.5.2011)
 277 #  - bugfix: fetch problem with space char in path, escape all params 
 278 #    containing non word chars
 279 #  - list available profiles, if given profile cannot be found
 280 #  - added --use-agent configuration hint
 281 #  - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke 
 282 #    fetch/restore
 283 #  - version command now prints out 'using installed' info
 284 #  - featreq 3166169: autotrust imported keys, based on code submitted by 
 285 #    Martin Ellis - imported keys are now automagically trusted ultimately 
 286 #  - new txt2man feature to create manpages for package maintainers
 287 #
 288 #  1.5.4.2 (6.1.2011)
 289 #  - new command changelog
 290 #  - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
 291 #  - bugfix: freebsd awk hangs on 'awk -W version' 
 292 #  - bugfix 3150244: mawk does not know '--version'
 293 #  - minor help text improvements
 294 #  - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
 295 #
 296 #  1.5.4.1 (4.12.2010)
 297 #  - output awk, python, bash version now in prolog
 298 #  - shebang uses /usr/bin/env now for freebsd compatibility, 
 299 #    bash not in /bin/bash 
 300 #  - new --disable-encryption parameter, 
 301 #    to override profile encr settings for one run
 302 #  - added exclude-if-present setting to conf template
 303 #  - bug 3126972: GPG_PW only needed for signing/symmetric encryption 
 304 #    (even though duplicity still needs it)
 305 #
 306 #  1.5.4 (15.11.2010)
 307 #  - as of 1.5.3 already, new ARCH_DIR config option
 308 #  - multiple key support
 309 #  - ftplicity-Feature Requests-2994929: separate encryption and signing key
 310 #  - key signing of symmetric encryption possible (duplicity patch committed)
 311 #  - gpg tests disable switch
 312 #  - gpg tests now previewable and more intelligent
 313 #
 314 #  1.5.3 (1.11.2010)
 315 #  - bugfix 3056628: improve busybox compatibility, grep did not have -m param
 316 #  - bugfix 2995408: allow empty password for PGP key
 317 #  - bugfix 2996459: Duply erroneously escapes '-' symbol in username
 318 #  - url_encode function is now pythonized
 319 #  - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
 320 #  - feature 3059262: Make pre and post aware of parameters, 
 321 #                     internal parameters + CMD of pre or post 
 322 #
 323 #  1.5.2.3 (16.4.2010)
 324 #  - bugfix: date again, should now work virtually anywhere
 325 #
 326 #  1.5.2.2 (3.4.2010)
 327 #  - minor bugfix: duplicity 0.6.8b version string now parsable
 328 #  - added INSTALL.txt
 329 #
 330 #  1.5.2.1 (23.3.2010)
 331 #  - bugfix: date formatting is awked now and should work on all platforms
 332 #
 333 #  1.5.2 (2.3.2010)
 334 #  - bugfix: errors print to STD_ERR now, failed tasks print an error message
 335 #  - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
 336 #  - simplified & cleaned profileless commands, removed second instance
 337 #  - generalized separator time routines
 338 #  - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
 339 #  - minor fixes
 340 #
 341 #  1.5.1.5 (5.2.2010)
 342 #  - bugfix: added special handling of credentials for rsync, imap(s)
 343 #
 344 #  1.5.1.4 (7.1.2010)
 345 #  - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
 346 #  - check if ncftp binary is available if url protocol is ftp
 347 #  - bugfix: duplicity output is now printed to screen directly to resolve
 348 #            'mem alloc problem' bug report
 349 #  - bugfix: passwords will not be in the url anymore to solve the 'duply shows
 350 #            sensitive data in process listing' bug report
 351 #
 352 #  1.5.1.3 (24.12.2009) 'merry xmas'
 353 #  - bugfix: gpg pass now apostrophed to allow space and friends
 354 #  - bugfix: credentials are now url encoded to allow special chars in them
 355 #            a note about url encoding has been added to the conf template
 356 #
 357 #  1.5.1.2 (1.11.2009)
 358 #  - bugfix: open parenthesis in password broke duplicity execution
 359 #  - bugfix: ssh/scp backend does not always need credentials e.g. key auth
 360 #
 361 #  1.5.1.1 (21.09.2009)
 362 #  - bugfix: fixed s3[+http] TARGET_PASS not needed routine
 363 #  - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
 364 #    see https://sourceforge.net/tracker/index.php?func=detail&
 365 #                aid=2864410&group_id=217745&atid=1041147
 366 #
 367 #  1.5.1 (21.09.2009) - duply (fka. ftplicity)
 368 #  - first things first: ftplicity (being able to support all backends since 
 369 #    some time) will be called duply (fka. ftplicity) from now on. The addendum
 370 #    is for the time being to circumvent confusion.
 371 #  - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity 
 372 #            command failed
 373 #  - s3[+http] now supported natively by translating user/pass to access_key/
 374 #    secret_key environment variables needed by duplicity s3 boto backend 
 375 #  - bugfix: additional output lines do not confuse version check anymore
 376 #  - list command supports now age parameter (patch by stefan on feature 
 377 #    request tracker)
 378 #  - bugfix: option/param pairs are now correctly passed on to duplicity
 379 #  - bugfix: s3[+http] needs no TARGET_PASS if command is read only
 380 #
 381 #  1.5.0.2 (31.07.1009)
 382 #  - bugfix: insert password in target url didn't work with debian mawk
 383 #            related to previous bug report
 384 #
 385 #  1.5.0.1 (23.07.2009)
 386 #  - bugfix: gawk gensub dependency raised an error on debian's default mawk
 387 #            replaced with match/substr command combination (bug report)
 388 #            https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
 389 #            group_id=217745
 390 #
 391 #  1.5.0 (01.07.2009)
 392 #  - removed ftp limitation, all duplicity backends should work now
 393 #  - bugfix: date for separator failed on openwrt busybox date, added a 
 394 #    detecting workaround, milliseconds are not available w/ busybox date
 395 #
 396 #  1.4.2.1 (14.05.2009)
 397 #  - bugfix: free temp space detection failed with lvm, fixed awk parse routine
 398 #
 399 #  1.4.2 (22.04.2009)
 400 #  - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
 401 #    armored ascii nature, the id helps if the key is switched for some reason
 402 #    im/export routines are updated accordingly (import is backward compatible 
 403 #    to the old profile/gpgkey files)         
 404 #  - profile argument is treated as path if it contains slashes 
 405 #    (for details see usage)
 406 #  - non-ftplicity options (all but --preview currently) are now passed 
 407 #    on to duplicity 
 408 #  - removed need for stat in secure_conf, it is ls based now
 409 #  - added profile folder readable check
 410 #  - added gpg version & home info output
 411 #  - awk utility availability is now checked, because it was mandatory already
 412 #  - tmp space is now checked on writability and space requirement
 413 #    test fails on less than 25MB or configured $VOLSIZE, 
 414 #    test warns if there is less than two times $VOLSIZE because 
 415 #    that's required for --asynchronous-upload option  
 416 #  - gpg functionality is tested now before executing duplicity 
 417 #    test drive contains encryption, decryption, comparison, cleanup
 418 #    this is meant to detect non trusted or other gpg errors early
 419 #  - added possibility of doing symmetric encryption with duplicity
 420 #    set GPG_KEY="" or simply comment it out
 421 #  - added hints in config template on the depreciation of 
 422 #    --short-filenames, --time-separator duplicity options
 423 #
 424 #  new versioning scheme 1.4.2b => 1.4.2, 
 425 #  beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
 426 #  to the first bug fixing version and 1.4.2.2 to the second and so on
 427 #  also the releases will now have a release date formatted (Day.Month.Year)
 428 #
 429 #  1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
 430 #            named exactly as the profile if existing in executing dir
 431 #          - improved plausibility checking of config and profile folder
 432 #          - secure_conf only acts if needed and prints a warning now
 433 #
 434 #  1.4.1b  - introduce status (duplicity collection-status) command
 435 #          - pre/post script output printed always now, not only on errors
 436 #          - new config parameter GPG_OPTS to pass gpg options
 437 #            added examples & comments to profile template conf
 438 #          - reworked separator times, added duration display
 439 #          - added --preview switch, to preview generated command lines
 440 #          - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
 441 #            profiles because they have reasonable defaults now if not set
 442 #
 443 #  1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
 444 #            therefore backup translates to pre_bkp_post now
 445 #          - bugfix: new command bkp, which represents duplicity's 
 446 #            default action (incr or full if full_if_older matches
 447 #            or no earlier backup chain is found)
 448 #
 449 #  new versioning scheme 1.4 => 1.4.0, added new minor revision number
 450 #  this is meant to slow down the rapid version growing but still keep 
 451 #  versions cleanly separated.
 452 #  only additional features will raise the new minor revision number. 
 453 #  all releases start as beta, each bugfix release will raise the beta 
 454 #  count, usually new features arrive before a version 'ripes' to stable
 455 #    
 456 #  1.4.0b
 457 #    1.4b  - added startup info on version, time, selected profile
 458 #          - added time output to separation lines
 459 #          - introduced: command purge-full implements duplicity's 
 460 #            remove-all-but-n-full functionality (patch by unknown),
 461 #            uses config variable $MAX_FULL_BACKUPS (default = 1)
 462 #          - purge config var $MAX_AGE defaults to 1M (month) now 
 463 #          - command full does not execute pre/post anymore
 464 #            use batch command pre_full_post if needed 
 465 #          - introduced batch mode cmd1_cmd2_etc
 466 #            (in turn removed the bvp command)
 467 #          - unknown/undefined command issues a warning/error now
 468 #          - bugfix: version check works with 0.4.2 and older now
 469 #    1.3b3 - introduced pre/post commands to execute/debug scripts
 470 #          - introduced bvp (backup, verify, purge)
 471 #          - bugfix: removed need for awk gensub, now mawk compatible
 472 #    1.3b2 - removed pre/post need executable bit set 
 473 #          - profiles now under ~/.ftplicity as folders
 474 #          - root can keep profiles in /etc/ftplicity, folder must be
 475 #            created by hand, existing profiles must be moved there
 476 #          - removed ftplicity in path requirement
 477 #          - bugfix: bash < v.3 did not know '=~'
 478 #          - bugfix: purge works again 
 479 #    1.3   - introduces multiple profiles support
 480 #          - modified some script errors/docs
 481 #          - reordered gpg key check import routine
 482 #          - added 'gpg key id not set' check
 483 #          - added error_gpg (adds how to setup gpg key howto)
 484 #          - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
 485 #          - duplicity_version_check routine introduced
 486 #          - added time separator, shortnames, volsize, full_if_older 
 487 #            duplicity options to config file (inspired by stevie 
 488 #            from http://weareroot.de) 
 489 #    1.1.1 - bugfix: encryption reactivated
 490 #    1.1   - introduced config directory
 491 #    1.0   - first release
 492 ################################################################################
 493 
 494 # utility functions overriding binaries
 495 
 496 # wrap grep to override possible env set GREP_OPTIONS=--color=always
 497 function grep {
 498   command env "GREP_OPTIONS=" grep "$@"
 499 }
 500 
 501 # implement basename in plain bash
 502 function basename {
 503   local stripped="${1%/}"
 504   echo "${stripped##*/}"
 505 }
 506 
 507 # implement dirname in plain bash
 508 function dirname {
 509   echo ${1%/*}
 510 }
 511 
 512 # implement basic which in plain bash
 513 function which {
 514   type -p "$@"
 515 }
 516 
 517 # check availability of executables via file name or file paths
 518 function lookup {
 519   local bin="$1"
 520   # look for file names in path via bash hash OR 
 521   # look for executables at given relative/absolute location
 522   ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
 523 }
 524 
 525 # important definitions #######################################################
 526 
 527 ME_LONG="$0"
 528 ME="$(basename $0)"
 529 ME_NAME="${ME%%.*}"
 530 ME_VERSION="2.4"
 531 ME_WEBSITE="https://duply.net"
 532 
 533 # default config values
 534 DEFAULT_SOURCE='/path/of/source'
 535 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
 536 DEFAULT_TARGET_USER='_backend_username_'
 537 DEFAULT_TARGET_PASS='_backend_password_'
 538 DEFAULT_GPG='gpg'
 539 DEFAULT_GPG_KEY='_KEY_ID_'
 540 DEFAULT_GPG_PW='_GPG_PASSWORD_'
 541 
 542 # function definitions ##########################
 543 
 544 function set_config { # sets global config vars
 545   local CONFHOME_COMPAT="$HOME/.ftplicity"
 546   local CONFHOME="$HOME/.duply"
 547   local CONFHOME_ETC_COMPAT="/etc/ftplicity"
 548   local CONFHOME_ETC="/etc/duply"
 549 
 550   # confdir can be delivered as path (must contain /)
 551   if [ `echo $FTPLCFG | grep /` ] ; then 
 552     CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
 553               ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
 554                && echo $(pwd)/${FTPLCFG} ) || \
 555               echo ${FTPLCFG})          
 556   # or DEFAULT in home/.duply folder (NEW)
 557   elif [ -d "${CONFHOME}" ]; then
 558     CONFDIR="${CONFHOME}/${FTPLCFG}"
 559   # or in home/.ftplicity folder (OLD)
 560   elif [ -d "${CONFHOME_COMPAT}" ]; then
 561     CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
 562     warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
 563   # root can put profiles under /etc/duply (NEW) if path exists
 564   elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
 565     CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
 566   # root can keep profiles under /etc/ftplicity (OLD) if path exists
 567   elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
 568     CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
 569     warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
 570   # hmm no profile folder there, then use default for error later
 571   else
 572     CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
 573   fi
 574 
 575   # remove trailing slash, get profile name etc.
 576   CONFDIR="${CONFDIR%/}"
 577   PROFILE="${CONFDIR##*/}"
 578   CONF="$CONFDIR/conf"
 579   PRE="$CONFDIR/pre"
 580   POST="$CONFDIR/post"
 581   EXCLUDE="$CONFDIR/exclude"
 582   KEYFILE="$CONFDIR/gpgkey.asc"
 583 }
 584 
 585 function version_info { # print version information
 586   cat <<END
 587   $ME_NAME version $ME_VERSION
 588   ($ME_WEBSITE)
 589 END
 590 }
 591 
 592 function version_info_using { 
 593   cat <<END
 594 $(version_info)
 595 
 596   $(using_info)
 597 END
 598 }
 599 
 600 function using_info {
 601   # init needed vars into global name space
 602   lookup duplicity && { duplicity_version_get; }
 603   local NOTFOUND="INVALID"
 604   local AWK_VERSION GREP_VERSION PYTHON_RUNNER \
 605     PYTHON_RUNNER_RESOLVED PYTHON_VERSION PYTHON_PATH
 606   # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
 607   AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 608   GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 609 
 610   if [ -n "$PYTHON" ]; then
 611     PYTHON_RUNNER=$PYTHON
 612   else
 613     PYTHON_RUNNER="$(duplicity_python_binary_parse)"
 614   fi
 615   # fetch version and resolve python
 616   [ -n "$PYTHON_RUNNER" ] && {
 617     PYTHON_VERSION=$($PYTHON_RUNNER -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
 618     local PYTHON_RUNNER_ARRAY=( $PYTHON_RUNNER )
 619     PYTHON_RUNNER_RESOLVED="$(which ${PYTHON_RUNNER_ARRAY[0]})"
 620     # readd params if there were
 621     [ ${#PYTHON_RUNNER_ARRAY[@]} -gt 1 ] && \
 622       PYTHON_RUNNER_RESOLVED="${PYTHON_RUNNER_RESOLVED} ${PYTHON_RUNNER_ARRAY[@]:1}"
 623       PYTHON_PATH="$($PYTHON_RUNNER -c "import sys;print(':'.join(sys.path));")"
 624   }
 625 
 626   local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
 627   local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
 628   # print out
 629   echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
 630 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($PYTHON_RUNNER_RESOLVED)}${PYTHON_PATH:+ 'PYTHONPATH=$PYTHON_PATH'}}\
 631 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
 632 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
 633 }
 634 
 635 function usage_info { # print usage information
 636 
 637   cat <<USAGE_EOF
 638 VERSION:
 639 $(version_info)
 640   
 641 DESCRIPTION: 
 642   Duply deals as a wrapper for the mighty duplicity magic.
 643   It simplifies running duplicity with cron or on command line by:
 644 
 645     - keeping recurring settings in profiles per backup job
 646     - enabling batch operations e.g. backup_verify+purge
 647     - executing pre/post scripts (different actions possible 
 648       depending on previous or next command or it's exit status)
 649     - precondition checking for flawless duplicity operation
 650 
 651   For each backup job one configuration profile must be created.
 652   The profile folder will be stored under '~/.${ME_NAME}/<profile>'
 653   (where ~ is the current users home directory).
 654   Hint:  
 655    If the folder '/etc/${ME_NAME}' exists, the profiles for the super
 656    user root will be searched & created there.
 657 
 658 USAGE:
 659   first time usage (profile creation):  
 660     $ME <profile> create
 661 
 662   general usage in single or batch mode (see EXAMPLES):  
 663     $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
 664 
 665   For batches the conditional separators can also be written as pseudo commands
 666   and(+), or(-). See SEPARATORS for details.
 667 
 668   Non $ME options are passed on to duplicity (see OPTIONS).
 669   All conf parameters can also be defined in the environment instead.
 670 
 671 PROFILE:
 672   Indicated by a path or a profile name (<profile>), which is resolved 
 673   to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
 674 
 675   Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
 676   the folder manually before running $ME_NAME as superuser.
 677   Note:  
 678     Already existing profiles in root's home folder will cease to work
 679     unless they are moved to the new location manually.
 680 
 681   example 1:   $ME humbug backup
 682 
 683   Alternatively a _path_ might be used e.g. useful for quick testing, 
 684   restoring or exotic locations. Shell expansion should work as usual.
 685   Hint:  
 686     The path must contain at least one path separator '/', 
 687     e.g. './test' instead of only 'test'.
 688 
 689   example 2:   $ME ~/.${ME_NAME}/humbug backup
 690 
 691 SEPARATORS:
 692   _ (underscore)  
 693              neutral separator
 694   + (plus sign), _and_  
 695              conditional AND
 696              the next command will only be executed if the previous succeeded
 697   - (minus sign), _or_  
 698              conditional OR
 699              the next command will only be executed if the previous failed
 700   [] (square brackets), _groupIn_/_groupOut_  
 701              enables grouping of commands
 702 
 703    example:  
 704     'pre+[bkp-verify]_post' translates to
 705     'pre_and_groupIn_bkp_or_verify_groupOut_post'
 706 
 707 COMMANDS:
 708   usage      get usage help text
 709 
 710   and/or/groupIn/groupOut  
 711              pseudo commands used in batches (see SEPARATORS above)
 712 
 713   create     creates a configuration profile
 714   backup     backup with pre/post script execution (batch: [pre_bkp_post]),
 715               full (if full_if_older matches or no earlier backup is found)
 716               incremental (in all other cases)
 717   pre/post   execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
 718   bkp        as above but without executing pre/post scripts
 719   full       force full backup
 720   incr       force incremental backup
 721   list [<age>]  
 722              list all files in backup (as it was at <age>, default: now)
 723   status     prints backup sets and chains currently in repository
 724   verify [<age>] [--compare-data]  
 725              list files changed, since age if given
 726   verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]  
 727              list changes of a file or folder path in backup compared to a
 728              local path, since age if given
 729   restore <target_path> [<age>]  
 730              restore the complete backup to <target_path> [as it was at <age>]
 731   fetch <src_path> <target_path> [<age>]  
 732              fetch single file/folder from backup [as it was at <age>]
 733   purge [<max_age>] [--force]  
 734              list outdated backup files (older than \$MAX_AGE)
 735               [use --force to actually delete these files]
 736   purgeFull [<max_full_backups>] [--force]  
 737              list outdated backup files (\$MAX_FULL_BACKUPS being the number of
 738              full backups and associated incrementals to keep, counting in 
 739              reverse chronological order)
 740               [use --force to actually delete these files]
 741   purgeIncr [<max_fulls_with_incrs>] [--force]  
 742              list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being 
 743              the number of full backups which associated incrementals will be
 744              kept, counting in reverse chronological order) 
 745               [use --force to actually delete these files]
 746   purgeAuto [--force]  
 747              convenience batch wrapper for all purge commands above.
 748              purge, purgeFull, purgeIncr are added if their conf vars were set. e.g.
 749               MAX_AGE=1Y
 750               MAX_FULL_BACKUPS=6
 751               MAX_FULLS_WITH_INCR=3
 752              in profile conf file would result in
 753               [purge_purgeFull_purgeIncr]
 754   cleanup [--force]  
 755              list broken backup chain files archives (e.g. after unfinished run)
 756               [use --force to actually delete these files]
 757 
 758   changelog  print changelog / todo list
 759   txt2man    feature for package maintainers - create a manpage based on the 
 760              usage output. download txt2man from http://mvertes.free.fr/, put 
 761              it in the PATH and run '$ME txt2man' to create a man page.
 762   version    show version information of $ME_NAME and needed programs
 763 
 764 OPTIONS:
 765   --force    passed to duplicity (see commands: 
 766              purge, purgeFull, purgeIncr, cleanup)
 767   --preview  do nothing but print out generated duplicity command lines
 768   --disable-encryption  
 769              disable encryption, overrides profile settings
 770 
 771 TIME FORMATS:
 772   For all time related parameters like age, max_age etc.
 773   Refer to the duplicity manpage for all available formats. Here some examples:
 774     2002-01-25T07:00:00+02:00 (full date time format string)
 775     2002/3/5 (date string YYYY/MM/DD)
 776     12D (interval, 12 days ago)
 777     1h78m (interval, 1 hour 78 minutes ago)
 778 
 779 PRE/POST SCRIPTS:
 780   Some useful internal duply variables are exported to the scripts.
 781 
 782     PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>, 
 783     GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
 784     CMD_<PREV|NEXT> (previous/next command), 
 785     CND_<PREV|NEXT> (condition before/after)
 786 
 787   The CMD_* variables were introduced to allow different actions according to 
 788   the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post' 
 789   will call the pre script two times, with CMD_NEXT variable set to 'bkp' 
 790   on the first and to 'verify' on the second run.
 791   CMD_ERR holds the exit code of the CMD_PREV .
 792 
 793 EXAMPLES:
 794   create profile 'humbug':  
 795     $ME humbug create (don't forget to edit this new conf file)
 796   backup 'humbug' now:  
 797     $ME humbug backup
 798   list available backup sets of profile 'humbug':  
 799     $ME humbug status
 800   list and delete outdated backups of 'humbug':  
 801     $ME humbug purge --force
 802   restore latest backup of 'humbug' to /mnt/restore:  
 803     $ME humbug restore /mnt/restore
 804   restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:  
 805     $ME humbug fetch etc/passwd /root/pw 4D
 806     (see "duplicity manpage", section TIME FORMATS)
 807   a one line batch job on 'humbug' for cron execution:  
 808     $ME humbug backup_verify_purge --force
 809   batch job to run a full backup with pre/post scripts:  
 810     $ME humbug pre_full_post
 811 
 812 FILES:
 813   in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
 814   conf             profile configuration file
 815   pre,post         pre/post scripts (see above for details)
 816   gpgkey.*.asc     exported GPG key files
 817   exclude          a globbing list of included or excluded files/folders
 818                    (see "duplicity manpage", section FILE SELECTION)
 819 
 820 $(hint_profile)
 821 
 822 SEE ALSO:
 823   duplicity man page:
 824     duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
 825 USAGE_EOF
 826 }
 827 
 828 # to check call 'duply txt2man | man -l -'
 829 function usage_txt2man {
 830   usage_info | \
 831   awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
 832   txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
 833 }
 834 
 835 function changelog {
 836   cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#(  )?/,"",$0);print}'
 837 }
 838 
 839 function create_config {
 840   if [ ! -d "$CONFDIR" ] ; then
 841     mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
 842   # create initial config file
 843     cat <<EOF >"$CONF"
 844 # gpg encryption settings, simple settings:
 845 #  GPG_KEY='disabled' - disables encryption alltogether
 846 #  GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
 847 #   sign if secret key of key1 is available use GPG_PW for sign & decrypt
 848 #  Note: you can specify keys via all methods described in gpg manpage,
 849 #        section "How to specify a user ID", escape commas (,) via backslash (\)
 850 #        e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
 851 #        as they are used to separate the entries
 852 #  GPG_PW='passphrase' - symmetric encryption using passphrase only
 853 GPG_KEY='${DEFAULT_GPG_KEY}'
 854 GPG_PW='${DEFAULT_GPG_PW}'
 855 # gpg encryption settings in detail (extended settings)
 856 #  the above settings translate to the following more specific settings
 857 #  GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
 858 #  GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
 859 #  GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
 860 #   encryption. If you want to deliver different passphrases for e.g. 
 861 #   several keys or symmetric encryption plus key signing you can use
 862 #   gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
 863 #   also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage 
 864 # notes on en/decryption
 865 #  private key and passphrase will only be needed for decryption or signing.
 866 #  decryption happens on restore and incrementals (compare archdir contents).
 867 #  for security reasons it makes sense to separate the signing key from the
 868 #  encryption keys. https://answers.launchpad.net/duplicity/+question/107216
 869 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
 870 #GPG_KEY_SIGN='<prvkey>'
 871 # set if signing key passphrase differs from encryption (key) passphrase
 872 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
 873 #GPG_PW_SIGN='<signpass>'
 874 
 875 # uncomment and set a file path or name force duply to use this gpg executable
 876 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
 877 #GPG='/usr/local/gpg-2.1/bin/gpg'
 878 
 879 # gpg options passed from duplicity to gpg process (default='')
 880 # e.g. "--trust-model pgp|classic|direct|always" 
 881 #   or "--compress-algo=bzip2 --bzip2-compress-level=9"
 882 #   or "--personal-cipher-preferences AES256,AES192,AES..."
 883 #   or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
 884 #   or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
 885 #      also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
 886 #GPG_OPTS=''
 887 
 888 # disable preliminary tests with the following setting
 889 #GPG_TEST='disabled'
 890 # disable automatic gpg key importing altogether
 891 #GPG_IMPORT='disabled'
 892 # disable automatic gpg key exporting to profile folder
 893 #GPG_EXPORT='disabled'
 894 
 895 # backend, credentials & location of the backup target (URL-Format)
 896 # generic syntax is
 897 #   scheme://[user[:password]@]host[:port]/[/]path
 898 # e.g.
 899 #   sftp://bob:secret@backupserver.com//home/bob/dupbkp
 900 # for details and available backends see duplicity manpage, section URL Format
 901 #   http://duplicity.nongnu.org/duplicity.1.html#sect7
 902 # BE AWARE:
 903 #   some backends (cloudfiles, S3 etc.) need additional env vars to be set to
 904 #   work properly, read after the TARGET definition for more details.
 905 # ATTENTION:
 906 #   characters other than A-Za-z0-9.-_.~ in the URL have to be
 907 #   replaced by their url encoded pendants, see
 908 #     http://en.wikipedia.org/wiki/Url_encoding
 909 #   if you define the credentials as TARGET_USER, TARGET_PASS below $ME
 910 #   will try to url_encode them for you if the need arises.
 911 TARGET='${DEFAULT_TARGET}'
 912 
 913 # optionally the username/password can be defined as extra variables
 914 # setting them here _and_ in TARGET results in an error
 915 # ATTENTION:
 916 #   there are backends that do not support the user/pass auth scheme.
 917 #   prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
 918 #   duplicity manpage. usually there is a NOTE section explaining if and which
 919 #   env vars should be set.
 920 #TARGET_USER='${DEFAULT_TARGET_USER}'
 921 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
 922 # e.g. for cloud files backend it might look like this (uncomment for use!)
 923 #export CLOUDFILES_USERNAME='someuser'
 924 #export CLOUDFILES_APIKEY='somekey'
 925 #export CLOUDFILES_AUTHURL ='someurl'
 926 # the following is an incomplete list (<backend>: comma separated env vars list)
 927 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
 928 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
 929 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
 930 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
 931 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
 932 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
 933 #        SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
 934 
 935 # base directory to backup
 936 SOURCE='${DEFAULT_SOURCE}'
 937 
 938 # a command that runs duplicity e.g. 
 939 #  shape bandwidth use via trickle
 940 #  "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
 941 #DUPL_PRECMD=""
 942 
 943 # override the python interpreter to execute duplicity, unset by default
 944 #  e.g. "python3" or "/usr/bin/python3.8"
 945 #PYTHON="python"
 946 
 947 # exclude folders containing exclusion file (since duplicity 0.5.14)
 948 # Uncomment the following two lines to enable this setting.
 949 #FILENAME='.duplicity-ignore'
 950 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
 951 
 952 # Time frame for old backups to keep, Used for the "purge" command.  
 953 # see duplicity man page, chapter TIME_FORMATS)
 954 #MAX_AGE=1M
 955 
 956 # Number of full backups to keep. Used for the "purgeFull" command. 
 957 # See duplicity man page, action "remove-all-but-n-full".
 958 #MAX_FULL_BACKUPS=1
 959 
 960 # Number of full backups for which incrementals will be kept for.
 961 # Used for the "purgeIncr" command.
 962 # See duplicity man page, action "remove-all-inc-of-but-n-full".
 963 #MAX_FULLS_WITH_INCRS=1
 964 
 965 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3) 
 966 # forces a full backup if last full backup reaches a specified age, for the 
 967 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
 968 # Uncomment the following two lines to enable this setting.
 969 #MAX_FULLBKP_AGE=1M
 970 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE " 
 971 
 972 # sets duplicity --volsize option (available since v0.4.3.RC7)
 973 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
 974 # VOLSIZE must be number of MB's to set the volume size to.
 975 # Uncomment the following two lines to enable this setting. 
 976 #VOLSIZE=50
 977 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
 978 
 979 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
 980 # default is 4, if not set
 981 #VERBOSITY=5
 982 
 983 # temporary file space. at least the size of the biggest file in backup
 984 # for a successful restoration process. (default is '/tmp', if not set)
 985 #TEMP_DIR=/tmp
 986 
 987 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
 988 # unencrypted meta data of the backup, enabling new incrementals without the 
 989 # need to decrypt backend metadata first. If empty or deleted somehow, the 
 990 # private key and it's password are needed.
 991 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
 992 #       big over time so you might want to put it not in the home dir.
 993 # default '~/.cache/duplicity/duply_<profile>/'
 994 # if set  '\${ARCH_DIR}/<profile>'
 995 #ARCH_DIR=/some/space/safe/.duply-cache
 996 
 997 # DEPRECATED setting
 998 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users 
 999 # to change the time separator from ':' to another character that will work 
1000 # on their system.  HINT: For Windows SMB shares, use --time-separator='_'.
1001 # NOTE: '-' is not valid as it conflicts with date separator.
1002 # ATTENTION: only use this with duplicity < 0.5.10, since then default file 
1003 #            naming is compatible and this option is pending depreciation 
1004 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
1005 
1006 # DEPRECATED setting
1007 # activates duplicity --short-filenames option, when uploading to a file
1008 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
1009 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
1010 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file 
1011 #            naming is compatible and this option is pending depreciation
1012 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
1013 
1014 # more duplicity command line options can be added in the following way
1015 # don't forget to leave a separating space char at the end
1016 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here " 
1017 
1018 EOF
1019 
1020 # create initial exclude file
1021     cat <<EOF >"$EXCLUDE"
1022 # although called exclude, this file is actually a globbing file list
1023 # duplicity accepts some globbing patterns, even including ones here
1024 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1025 # - dir/bar/foo
1026 # + dir/bar
1027 # - **
1028 # for more details see duplicity manpage, section File Selection
1029 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1030 
1031 EOF
1032 
1033   # Hints on first usage
1034   cat <<EOF
1035 
1036 Congratulations. You just created the profile '$FTPLCFG'.
1037 The initial config file has been created as 
1038 '$CONF'.
1039 You should now adjust this config file to your needs.
1040 
1041 $(hint_profile)
1042 
1043 EOF
1044 fi
1045 
1046 }
1047 
1048 # used in usage AND create_config
1049 function hint_profile {
1050   cat <<EOF
1051 IMPORTANT:
1052   Copy the _whole_ profile folder after the first backup to a safe place.
1053   It contains everything (duply related) needed to restore your backups. 
1054 
1055   Pay attention to (possibly later added) external files such as credentials
1056   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1057   (e.g. DPBX_ACCESS_TOKEN). 
1058   It is good policy to place those in the profile folder if possible at all.
1059     e.g. in case of 'multi://' target the config .json file
1060   Env vars should be added to duply profiles' conf file.
1061 
1062   Keep access to these files restricted as they contain information (gpg key,
1063   passphrases etc.) to access and modify your backups.
1064   
1065   Finally:
1066   You should attempt a restore from an unrelated host to be sure you really
1067   have everything needed for restoration.
1068 
1069   Repeat these steps after _all_ configuration changes. Some configuration 
1070   options are crucial for restoration.
1071 
1072 EOF
1073 }
1074 
1075 function separator {
1076   echo "--- $@ ---"
1077 }
1078 
1079 function inform {
1080   echo -e "\nINFO:\n\n$@\n"
1081 }
1082 
1083 function warning {
1084   echo -e "\nWARNING:\n\n$@\n"
1085 }
1086 
1087 function warning_oldhome {
1088   local old=$1 new=$2
1089   warning " ftplicity changed name to duply since you created your profiles.
1090   Please rename the old folder
1091   '$old'
1092   to
1093   '$new'
1094   and this warning will disappear.
1095   If you decide not to do so profiles will _only_ work from the old location."
1096 }
1097 
1098 function error_print {
1099   echo -e "$@" >&2
1100 }
1101 
1102 function error {
1103   error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1104   exit -1
1105 }
1106 
1107 function error_gpg {
1108   [ -n "$2" ] && local hint="\n  $2\n\n  "
1109   
1110   error "$1
1111 
1112 Hint${hint:+s}:
1113   ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1114   Don't forget the used _password_ as you will need it.
1115   When done enter the 8 digit id & the password in the profile conf file.
1116 
1117   The key id can be found doing a 'gpg --list-keys'. In the example output 
1118   below the key id for the public key would be FFFFFFFF.
1119 
1120   pub   1024D/FFFFFFFF 2007-12-17
1121   uid                  duplicity
1122   sub   2048g/899FE27F 2007-12-17
1123 "
1124 }
1125 
1126 function error_gpg_test {
1127   [ -n "$2" ] && local hint="\n  $2\n\n  "
1128 
1129   error "$1
1130 
1131 Hint${hint:+s}:
1132   ${hint}This error means that gpg is probably misconfigured or not working 
1133   correctly. The error message above should help to solve the problem.
1134   However, if for some reason $ME_NAME should misinterpret the situation you 
1135   can define GPG_TEST='disabled' in the conf file to bypass the test.
1136   Please do not forget to report the bug in order to resolve the problem
1137   in future versions of $ME_NAME.
1138 "
1139 }
1140 
1141 function error_path {
1142   error "$@
1143 PATH='$PATH'
1144 "
1145 }
1146 
1147 function error_to_string {
1148   [ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1149 }
1150 
1151 function duplicity_version_get {
1152   # use cached value, just print
1153   var_isset DUPL_VERSION && return
1154 
1155   local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1156   # only run with a user specific python if configured (running by default
1157   # breaks homebrew as they place a shell wrapper for duplicity in path)
1158   [ -n "$PYTHON" ] &&\
1159     CMD="$PYTHON $(qw "$(which duplicity)")"
1160 
1161   DUPL_VERSION_OUT=$($CMD --version)
1162   DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1163   #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1164   DUPL_VERSION_VALUE=0
1165   DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1166   if (match(v,/[^\.0-9]+[0-9]*$/)){
1167     rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1168   if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1169   split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1170   print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1171   print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1172   print "DUPL_VERSION_RC=\047"rc"\047"
1173   print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1174   }')
1175   eval "$DUPL_VERSION_AWK"
1176   #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1177 
1178   # doublecheck findings and report error
1179   if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1180     inform "duplicity version check failed (please report, this is a bug)
1181 the command
1182   $CMD
1183 resulted in
1184   $DUPL_VERSION_OUT
1185 "
1186   elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1187     error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1188 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1189 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1190   fi
1191 }
1192 
1193 function duplicity_version_ge {
1194   [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1195 }
1196 
1197 function duplicity_version_lt {
1198   ! duplicity_version_ge "$1"
1199 }
1200 
1201 # parse interpreter from duplicity shebang
1202 function duplicity_python_binary_parse {
1203   # reuse cached result
1204   var_isset 'DUPL_PYTHON_BIN' && {
1205     [ -n "$DUPL_PYTHON_BIN" ] && {
1206       echo $DUPL_PYTHON_BIN
1207       return
1208     } || return 1
1209   }
1210 
1211   local DUPL_BIN=$(which duplicity)
1212   # test for shebang
1213   IFS= LC_ALL=C read -rN2 shebang < "$DUPL_BIN" && [ "$shebang" != '#!' ] && {
1214     DUPL_PYTHON_BIN=""
1215     return 1
1216   }
1217 
1218   # parse it or warn
1219   DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1220   if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1221     warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was 
1222 '$DUPL_PYTHON_BIN'.
1223 "
1224     DUPL_PYTHON_BIN=""
1225     return 1
1226   fi
1227   
1228   # success
1229   echo $DUPL_PYTHON_BIN
1230   return
1231 }
1232 
1233 function run_script { # run pre/post scripts
1234   local ERR=0
1235   local SCRIPT="$1"
1236   if [ ! -z "$PREVIEW" ] ; then
1237     echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1238   elif [ -r "$SCRIPT" ] ; then 
1239     echo -n "Running '$SCRIPT' "
1240     if [ -x "$SCRIPT" ]; then
1241       OUT=$("$SCRIPT" 2>&1)
1242       ERR=$?
1243     else
1244       OUT=$(. "$SCRIPT" 2>&1)
1245       ERR=$?
1246     fi
1247     [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1248     echo -en ${OUT:+"Output: $OUT\n"} ;
1249   else
1250     echo "Skipping n/a script '$SCRIPT'."
1251   fi
1252   return $ERR
1253 }
1254 
1255 function run_cmd {
1256   # run or print escaped cmd string
1257   local CMD_ERR=0
1258   if [ -n "$PREVIEW" ]; then
1259     CMD_OUT=$( echo "$@ 2>&1" )
1260     CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1261   elif [ -n "$CMD_DISABLED" ]; then
1262     CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1263   else
1264     echo -n -e "$CMD_MSG"
1265     CMD_OUT=` eval "$@" 2>&1 `
1266     CMD_ERR=$?
1267     if [ "$CMD_ERR" = "0" ]; then
1268       CMD_MSG=" (OK)"
1269     else
1270       CMD_MSG=" (FAILED)"
1271     fi
1272   fi
1273   echo -e "$CMD_MSG"
1274   # reset
1275   unset CMD_DISABLED CMD_MSG
1276   return $CMD_ERR
1277 }
1278 
1279 function qw { quotewrap "$@"; }
1280 
1281 function quotewrap {
1282   local param="$@"
1283   # quote strings having non word chars (e.g. spaces)
1284   if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1285     echo "$param" | awk '{\
1286       gsub(/[\047]/,"\047\\\047\047",$0);\
1287       gsub(/[\042]/,"\047\\\042\047",$0);\
1288       print "\047"$0"\047"}'
1289     return
1290   fi
1291   echo $param
1292 }
1293 
1294 function duplicity_params_global {
1295   # already done? return
1296   var_isset 'DUPL_PARAMS_GLOBAL' && return
1297   local DUPL_ARG_ENC
1298 
1299   # use key only if set in config, else leave it to symmetric encryption
1300   if gpg_disabled; then
1301     local DUPL_PARAM_ENC='--no-encryption'
1302   else
1303     local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1304     gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1305     # interpret password settings
1306     var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1307     var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1308   fi
1309 
1310   local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1311 
1312   # set name for dupl archive folder, since 0.6.0
1313   if duplicity_version_ge 601; then
1314     local DUPL_ARCHDIR=''
1315     if var_isset 'ARCH_DIR'; then
1316       DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1317       # reuse erronously duply_ prefixed folders from bug #117
1318       if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1319         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1320       else
1321         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1322       fi
1323     else
1324       DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1325     fi
1326   fi
1327 
1328 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1329 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1330  ${GPG_OPTS}"
1331 
1332 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1333  ${DUPL_ARG_ENC}"
1334 }
1335 
1336 # function to filter the DUPL_PARAMS var from user conf
1337 function duplicity_params_conf {
1338   # reuse cmd var from main loop
1339   ## in/exclude parameters are currently not supported on restores
1340   if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1341     # filter exclude params from fetch/restore/status
1342     eval "stripXcludes $DUPL_PARAMS"
1343     return
1344   fi
1345 
1346   # nothing done, print unchanged
1347   echo "$DUPL_PARAMS"
1348 }
1349 
1350 # strip in/exclude parameters from param string
1351 function stripXcludes {
1352   local STRIPNEXT OUT;
1353   for p in "$@"; do
1354     if [ -n "$STRIPNEXT" ]; then
1355       unset STRIPNEXT
1356       # strip the value of previous parameter
1357       continue
1358     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1359       # strips e.g. --include /foo/bar
1360       STRIPNEXT="yes"
1361       continue
1362     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1363       # strips e.g. --include=/foo/bar
1364       continue
1365     fi
1366     
1367     OUT="$OUT $(qw "$p")"
1368   done
1369   echo "$OUT"
1370 }
1371 
1372 function duplify { # the actual wrapper function
1373   local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1374 
1375   # put command (with params) first in duplicity parameters
1376   for param in "$@" ; do
1377   # split cmd from params (everything before splitchar --)
1378     if [ "$param" == "--" ] ; then
1379       PARAMSNOW=1
1380     else
1381       # wrap in quotes to protect from spaces
1382       [ ! $PARAMSNOW ] && \
1383         DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1384       || \
1385         DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1386     fi
1387   done
1388 
1389   # init global duplicity parameters same for all tasks
1390   duplicity_params_global
1391 
1392   local RUN=eval CMD=duplicity
1393   # run in cmd line preview mode if requested
1394   var_isset 'PREVIEW' && RUN=echo
1395   # only run with a user specific python if configured (running by default
1396   # breaks homebrew as they place a shell wrapper for duplicity in path)
1397   # resolve duplicity path for usage with python interpreter
1398   [ -n "$PYTHON" ] &&\
1399     CMD="$PYTHON $(qw "$(which duplicity)")"
1400 
1401 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1402   ${DUPL_PRECMD} $CMD $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1403   $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1404 
1405   local ERR=$?
1406   return $ERR
1407 }
1408 
1409 function secureconf { # secure the configuration dir
1410   #PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1411   local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1412   if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1413     chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1414     warning "The profile's folder 
1415 '$CONFDIR'
1416 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1417   fi
1418 }
1419 
1420 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1421 function date_fix {
1422   local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1423   local date
1424   #[ "$1" == "%N" ] && return #test the no nsec test below
1425   # gnu date with -d @epoch
1426   date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1427     echo $date && return
1428   # date bsd,osx with -r epoch
1429   date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1430     echo $date && return
1431   # date busybox with -d epoch -D %s
1432   date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1433     echo $date && return
1434   ## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1435   # python fallback
1436   #date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1437   #  echo $date && return
1438   # awk fallback
1439   date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1440     echo $date && return
1441   # perl fallback
1442   date=$(perl  -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1443     echo $date && return
1444   # error
1445   echo "ERROR"
1446   return 1
1447 }
1448 
1449 function nsecs {
1450   local NSECS
1451   # test if date supports nanosecond output
1452   if ! var_isset NSECS_DISABLED; then
1453     NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1454     [ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1455   fi
1456 
1457   # add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1458   if [ "$NSECS_DISABLED" == "1" ]; then
1459     date_fix %s000000000
1460   else
1461     date_fix %s%N
1462   fi
1463 }
1464 
1465 function nsecs_to_sec {
1466   echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1467 }
1468 
1469 function datefull_from_nsecs {
1470   date_from_nsecs $1 '%F %T'
1471 }
1472 
1473 function date_from_nsecs {
1474   local FORMAT=${2:-%T}
1475   local TIME=$(nsecs_to_sec $1)
1476   local SECS=${TIME%.*}
1477   local DATE=$(date_fix "${FORMAT}" ${SECS:-0})
1478   echo $DATE.${TIME#*.}
1479 }
1480 
1481 function var_isset {
1482   if [ -z "$1" ]; then
1483     echo "ERROR: function var_isset needs a string as parameter"
1484   elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1485     return 0
1486   fi
1487   return 1
1488 }
1489 
1490 function is_condition {
1491   local CMD=$(tolower "$1")
1492   [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1493 }
1494 
1495 function is_groupMarker {
1496   local CMD=$(tolower "$1")
1497   [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1498 }
1499 
1500 function is_command {
1501   local CMD=$(tolower "$1")
1502   ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1503 }
1504 
1505 function url_encode {
1506   # utilize python, silently do nothing on error - because no python no duplicity
1507   local PYTHON_RUNNER
1508   PYTHON_RUNNER="$(duplicity_python_binary_parse)" ||\
1509   PYTHON_RUNNER="python" &&\
1510     OUT=$($PYTHON_RUNNER -c "
1511 try: import urllib.request as urllib
1512 except ImportError: import urllib
1513 print(urllib.${2}quote('$1'));
1514 " 2>/dev/null ); ERR=$?
1515   [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1516 }
1517 
1518 function url_decode {
1519   # reuse function above with a simple string param hack
1520   url_encode "$1" "un"
1521 }
1522 
1523 function toupper {
1524   echo "$@"|awk '$0=toupper($0)'
1525 }
1526 
1527 function tolower {
1528   echo "$@"|awk '$0=tolower($0)'
1529 }
1530 
1531 function isnumber {
1532   case "$*" in
1533     ''|*[!0-9]*) return 1;;
1534     *) return 0;;
1535   esac
1536 }
1537 
1538 #function tmp_space {
1539 #  
1540 #  if ! isnumber $VOLSIZE; then
1541 #    inform "failed to determine free space (please report, this is a bug)"
1542 #    return
1543 #  fi
1544 #  
1545 # get free temp space
1546 #  TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1547 #  # check for free space or FAIL
1548 #  if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1549 #    error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1550 #than one duplicity volume (${VOLSIZE}MB).
1551 #    
1552 #  Hint: Free space or change TEMP_DIR setting."
1553 #fi
1554 #
1555 #}
1556 
1557 function gpg_disabled {
1558   echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1559 }
1560 
1561 # usage: join SEPARATOR "entry1" "entry2"
1562 function join {
1563   local SEP="$1" ENTRY OUT; shift;
1564   for ENTRY in "$@"; do
1565     ENTRY=${ENTRY//$SEP/\\$SEP}
1566     [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1567   done
1568   echo $OUT
1569 }
1570 
1571 function gpg_testing {
1572   [ "$GPG_TEST" != "disabled" ]
1573 }
1574 
1575 function gpg_signing {
1576   echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1577 }
1578 
1579 function gpg_keytype {
1580   echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1581 }
1582 
1583 # parameter key id, key_type
1584 function gpg_keyfile {
1585   local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1586   local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1587   echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1588 }
1589 
1590 # parameter key id
1591 function gpg_import {
1592   local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1593   [ "$GPG_IMPORT" = "disabled" ] && {
1594     echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1595     return
1596   }
1597 
1598   # create a list of legacy key file names and current naming scheme
1599   # we always import pub and sec if they are avail in conf folder
1600   local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1601                    $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1602 
1603   # Try autoimport from existing old gpgkey files 
1604   # and new gpgkey.XXX.asc files (since v1.4.2)
1605   # and even newer gpgkey.XXX.[pub|sec].asc
1606   for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1607     FILE=${KEYFILES[$i]}
1608     if [ -f "$FILE" ]; then
1609       FOUND=1
1610       
1611       CMD_MSG="Import keyfile '$FILE' to keyring"
1612       run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1613       if [ "$?" != "0" ]; then 
1614         warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1615         ERR=1
1616         # continue with next
1617         continue
1618       fi
1619     fi
1620   done
1621 
1622   if [ "$FOUND" -eq 0 ]; then
1623     echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1624     return 1
1625   fi
1626 
1627   # try to set trust automagically
1628   CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1629   run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1630   if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then 
1631    # success on all levels, we're done
1632    return $ERR
1633   fi
1634 
1635   # failover: user has to set trust manually
1636   echo -e "For $ME_NAME to work you have to set the trust level 
1637 with the command \"trust\" to \"ultimate\" (5) now.
1638 Exit the edit mode of gpg with \"quit\"."
1639   CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1640   run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1641 
1642   return $ERR
1643 }
1644 
1645 # see 'How to specify a user ID' on gpg manpage
1646 function gpg_fingerprint {
1647   gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1648   awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1649 }
1650 
1651 function gpg_export_if_needed {
1652   [ "$GPG_EXPORT" = 'disabled' ] && { \
1653     echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1654     return
1655   }
1656 
1657   local SUCCESS FILE KEY_TYPE
1658   local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1659   for KEY_ID in "$@"; do
1660     # check if already exported, do it if not
1661     for KEY_TYPE in PUB SEC; do
1662       FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1663       if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1664 
1665         # exporting
1666         CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1667         # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1668         # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1669         run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1670         CMD_ERR=$?
1671 
1672         if [ "$CMD_ERR" = "0" ]; then
1673           CMD_MSG="Write file '"$(basename "$FILE")"'"
1674           run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1675         fi
1676 
1677         if [ "$CMD_ERR" != "0" ]; then
1678           warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1679         else
1680           SUCCESS=1
1681         fi
1682 
1683         # cleanup
1684         rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1685       fi
1686     done
1687   done
1688   
1689   [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1690 You should backup your changed profile folder now and store it in a safe place."
1691 }
1692 
1693 # replace all non-alnum chars with underscore (for file operations)
1694 function gpg_key_legalize {
1695   echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1696 }
1697 
1698 function gpg_key_cache {
1699   local RES
1700   local MODE=$1
1701   shift
1702   local PREFIX="GPG_KEY"
1703   local SUFFIX=$(gpg_key_legalize "$@")
1704   local KEYID="$*"
1705   local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1706   if [ "$MODE" = "RESET" ]; then
1707     eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1708     return 255
1709   elif ! var_isset "$CACHE"; then
1710     if [ "$MODE" = "PUB" ]; then
1711       RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1712     elif [ "$MODE" = "SEC" ]; then
1713       RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1714     else
1715       return 255
1716     fi
1717     eval $CACHE=$RES
1718   fi
1719   eval return \$$CACHE
1720 }
1721 
1722 function gpg_pub_avail {
1723   gpg_key_cache PUB "$@"
1724 }
1725 
1726 function gpg_sec_avail {
1727   gpg_key_cache SEC "$@"
1728 }
1729 
1730 function gpg_key_format {
1731   echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1732 }
1733 
1734 # splits a comma separated line into lines, respects escaped commas
1735 function gpg_split_keyset {
1736   local LIST
1737   LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1738   echo -e "$LIST"
1739 }
1740 
1741 function gpg_prefix_keyset {
1742   local PREFIX="$1" OUT=""
1743   shift
1744   for KEY_ID in "$@"; do
1745     OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1746   done
1747   echo $OUT
1748 }
1749 
1750 # grep a variable from conf text file (currently not used)
1751 function gpg_passwd {
1752   [ -r "$CONF" ] && \
1753   awk '/^[ \t]*GPG_PW[ \t=]/{\
1754         sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1755         gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1756         print $0; exit}' "$CONF"
1757 }
1758 
1759 # return success if at least one secret key is available
1760 function gpg_key_decryptable {
1761   # no keys, no problem
1762   gpg_symmetric && return 0
1763 
1764   local KEY_ID
1765   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1766     gpg_sec_avail "$KEY_ID" && return 0
1767   done
1768   return 1
1769 }
1770 
1771 function gpg_symmetric {
1772   [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1773 }
1774 
1775 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1776 function gpg_param_passwd {
1777   var_isset GPG_USEAGENT && exit 1
1778   
1779   if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1780     echo "--passphrase-fd 0 --batch"
1781   fi
1782 }
1783 
1784 # select the earliest defined and create an "echo <value> |" string
1785 function gpg_pass_pipein {
1786   var_isset GPG_USEAGENT && exit 1
1787   
1788   for var in "$@"
1789   do
1790     if var_isset "$var"; then
1791       echo "echo $(qw $(eval echo \$$var)) |"
1792       return 0
1793     fi
1794   done
1795   
1796   return 1
1797 }
1798 
1799 # checks if gpg-agent is available, returns error code
1800 # 0 on success
1801 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1802 # 2 if GPG_AGENT_INFO is stale
1803 # 3 cannot connect to gpg-agent
1804 function gpg_agent_avail {
1805   # GPG_AGENT_INFO is deprecated in gpg2.1, 
1806   # first try to connect to a possibly running agent here
1807   local ERR=3
1808   gpg-agent > /dev/null 2>&1 && return 0
1809 
1810   # detect stale pid in legacy GPG_AGENT_INFO env var
1811   if var_isset GPG_AGENT_INFO; then
1812     # check if a pid matching process is running at all
1813     local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1814     if isnumber "$GPG_AGENT_PID"; then
1815       ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1816     fi
1817   fi
1818 
1819   return $ERR
1820 }
1821 
1822 function gpg_custom_binary {
1823   var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1824     echo "--gpg-binary $(qw "$GPG")"
1825 }
1826 
1827 function gpg_binary {
1828   local BIN
1829   var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1830   echo "$BIN"
1831 }
1832 
1833 function gpg_avail {
1834   lookup "$(gpg_binary)"
1835 }
1836 
1837 # enforce the use of our selected gpg binary
1838 function gpg {
1839   command "$(gpg_binary)" "$@"
1840 }
1841 export -f gpg
1842 
1843 # start of script #######################################################################
1844 
1845 # confidentiality first, all we create is only readable by us
1846 umask 077
1847 
1848 # check if ftplicity is there & executable
1849 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1850 
1851 if [ ${#@} -eq 1 ]; then
1852   cmd="${1}"
1853 else
1854   FTPLCFG="${1}" ; cmd="${2}"
1855 fi
1856 
1857 # deal with command before profile validation calls
1858 # show requested version
1859 # OR requested usage info
1860 # OR create a profile
1861 # OR fall through
1862 ##if [ ${#@} -le 2 ]; then
1863 case "$cmd" in
1864   changelog)
1865     changelog
1866     exit 0
1867     ;;
1868   create)
1869     set_config
1870     if [ -d "$CONFDIR" ]; then
1871       error "The profile '$FTPLCFG' already exists in
1872 '$CONFDIR'.
1873 
1874 Hint:
1875  If you _really_ want to create a new profile by this name you will 
1876  have to manually delete the existing profile folder first."
1877       exit 1
1878     else
1879       create_config
1880       exit 0
1881     fi
1882     ;;
1883   txt2man)
1884     set_config
1885     usage_txt2man
1886     exit 0
1887     ;;
1888   usage|-help|--help|-h|-H)
1889     set_config
1890     usage_info
1891     exit 0
1892     ;;
1893   version|-version|--version|-v|-V)
1894     # profile can override GPG/PYTHON, so import it if it was given
1895     var_isset FTPLCFG && {
1896       set_config
1897       [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1898     }
1899     version_info_using
1900     exit 0
1901     ;;
1902   # fallthrough.. we got a command that needs an existing profile
1903   *)
1904     # if we reach here, user either forgot profile or chose wrong profileless command
1905     if [ ${#@} -le 1 ]; then
1906       error "\
1907  Missing or wrong parameters. 
1908  Only the commands 
1909    changelog, create, usage, txt2man, version
1910  can be called without selecting an existing profile first.
1911  Your command was '$cmd'.
1912 
1913  Hint: Run '$ME usage' to get help."
1914     fi
1915 esac
1916 
1917 
1918 # Hello world
1919 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1920 
1921 # check system environment
1922 
1923 # is duplicity avail
1924 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1925 # init, exec duplicity version check info
1926 duplicity_version_get
1927 
1928 # check for certain important helper programs
1929 # TODO: we should probably check for duplicity and $PYTHON (if set) here too
1930 for f in awk grep ; do
1931   lookup "$f" || \
1932     error_path "$f missing. installed und available in path?"
1933 done
1934 
1935 ### read configuration
1936 set_config
1937 # check validity
1938 if [ ! -d "$CONFDIR" ]; then 
1939     error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1940 '$CONFDIR'.
1941 
1942 Hints:
1943 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1944  Use '$ME <name> create' to create a new profile.
1945  Use '$ME usage' to get usage help."
1946 elif [ ! -x "$CONFDIR" ]; then
1947     error "\
1948 Profile folder in '$CONFDIR' cannot be accessed.
1949 
1950 Hint: 
1951  Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1952 elif [ ! -f "$CONF" ] ; then
1953   error "'$CONF' not found."
1954 elif [ ! -r "$CONF" ] ; then
1955   error "'$CONF' not readable."
1956 else
1957   . "$CONF"
1958   #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1959   TEMP_DIR=${TEMP_DIR:-'/tmp'}
1960   # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1961   if var_isset 'TARGET_PW'; then
1962     TARGET_PASS="${TARGET_PW}"
1963   fi
1964 fi
1965 echo "Using profile '$CONFDIR'."
1966 
1967 # secure config dir, if needed w/ warning
1968 secureconf
1969 
1970 # split TARGET in handy variables
1971 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1972   target=$0; match(target,/^([^\/:]+):\/\//); \
1973   prot=substr(target,RSTART,RLENGTH);\
1974   rest=substr(target,RSTART+RLENGTH); \
1975   if (credsavail=match(rest,/^[^@]*@/)){\
1976     creds=substr(rest,RSTART,RLENGTH-1);\
1977     credcount=split(creds,cred,":");\
1978     rest=substr(rest,RLENGTH+1);\
1979     # split creds with regexp\
1980     match(creds,/^([^:]+)/);\
1981     user=substr(creds,RSTART,RLENGTH);\
1982     pass=substr(creds,RSTART+1+RLENGTH);\
1983   };\
1984   # filter quotes or escape them\
1985   gsub(/[\047\042]/,"",prot);\
1986   gsub(/[\047\042]/,"",rest);\
1987   gsub(/[\047]/,"\047\\\047\047",creds);\
1988   print "TARGET_URL_PROT=\047"prot"\047\n"\
1989          "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1990          "TARGET_URL_CREDS=\047"creds"\047\n";\
1991    if(user){\
1992      gsub(/[\047]/,"\047\\\047\047",user);\
1993      print "TARGET_URL_USER=\047"user"\047\n"}\
1994    if(pass){\
1995      gsub(/[\047]/,"\047\\\047\047",pass);\
1996      print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1997   }')
1998 eval "${TARGET_SPLIT_URL}"
1999 
2000 # fetch commmand from parameters ########################################################
2001 # Hint: cmds is also used to check if authentification info sufficient in the next step 
2002 cmds="$2"; shift 2
2003 
2004 # complain if command(s) missing
2005 [ -z $cmds ] && error "  No command given.
2006 
2007   Hint: 
2008     Use '$ME usage' to get usage help."
2009 
2010 # process params
2011 for param in "$@"; do
2012   #echo !$param!
2013   case "$param" in
2014     # enable ftplicity preview mode
2015     '--preview')
2016       PREVIEW=1
2017       ;;
2018     # interpret duplicity disable encr switch
2019     '--disable-encryption')
2020       GPG_KEY='disabled'
2021       ;;
2022     *)
2023       if [ `echo "$param" | grep -e "^-"` ] || \
2024          [ `echo "$last_param" | grep -e "^-"` ] ; then
2025         # forward parameter[/option pairs] to duplicity
2026         dupl_opts["${#dupl_opts[@]}"]=${param}
2027       else
2028         # anything else must be a parameter (e.g. for fetch, ...)
2029         ftpl_pars["${#ftpl_pars[@]}"]=${param}
2030       fi
2031       last_param=${param}
2032       ;;
2033   esac
2034 done
2035 
2036 # plausibility check config - VARS & KEY ################################################
2037 # check if src, trg, trg pw
2038 # auth info sufficient 
2039 # gpg key, gpg pwd (might be empty) set in config
2040 # OR key in local gpg db
2041 # OR key can be imported from keyfile 
2042 # OR fail
2043 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
2044  error " Source Path (setting SOURCE) not set or still default value in conf file 
2045  '$CONF'."
2046 
2047 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2048  error " Backup Target (setting TARGET) not set or still default value in conf file 
2049  '$CONF'."
2050 
2051 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2052      [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2053  error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}') 
2054  are configured with different values. There can be only one.
2055  
2056  Hint: Remove conflicting setting."
2057 
2058 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2059      [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2060  error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}') 
2061  are configured with different values. There can be only one.
2062  
2063  Hint: Remove conflicting setting."
2064 fi
2065 
2066 # GPG config plausibility check1 (disabled check) #############################
2067 if gpg_disabled; then
2068   : # encryption disabled, all is well
2069 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2070   warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file 
2071 '$CONF'.
2072 Will disable encryption for duplicity now.
2073 
2074 Hint: 
2075  If you really want to use _no_ encryption you can disable this warning by 
2076  setting GPG_KEY='disabled' in conf file."
2077   GPG_KEY='disabled'
2078 fi
2079 
2080 # GPG availability check (now we know if gpg is really needed)#################
2081 if ! gpg_disabled; then 
2082   gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2083 fi
2084 
2085 # Output versions info ########################################################
2086 using_info
2087 
2088 # GPG create key settings, config check2 (needs gpg) ##########################
2089 if gpg_disabled; then
2090   : # the following tests are not necessary
2091 else
2092 
2093 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2094 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2095 
2096 # enc key still default?
2097 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then 
2098   error_gpg "Encryption Key GPG_KEY still default in conf file 
2099 '$CONF'."
2100 fi
2101 
2102 # create array of gpg encr keys, for further processing
2103 OIFS="$IFS" IFS=$'\n'
2104 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2105 IFS="$OIFS"
2106 
2107 # pw set? 
2108 # symmetric needs one, always
2109 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2110   ; then
2111   error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) 
2112 is empty/not set or still default value in conf file 
2113 '$CONF'."
2114 fi
2115 # this is a technicality, we can only pump one pass via pipe into gpg
2116 # but symmetric already always needs one for encryption
2117 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2118   [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2119   error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2120 '$CONF'.
2121 This is unfortunately impossible. For details see duplicity manpage, 
2122 section 'A Note On Symmetric Encryption And Signing'.
2123 
2124 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2125 Tip2: Use gpg-agent."
2126 fi
2127 
2128 # test - GPG KEY AVAILABILITY ##################################################
2129 
2130 # check gpg public keys availability, try import if needed
2131 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2132   KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2133   # test availability, try to import, retest
2134   if ! gpg_pub_avail "${KEY_ID}"; then
2135     echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2136     gpg_import "${KEY_ID}" PUB
2137     gpg_key_cache RESET "${KEY_ID}"
2138     gpg_pub_avail "${KEY_ID}" || { \
2139       gpg_testing && error_gpg \
2140       "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2141       "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2142   as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2143   If not you can put it there and $ME_NAME will autoimport it on the next run.
2144   Alternatively import it manually as the user you plan to run $ME_NAME with."
2145     }
2146   else
2147     echo "Public key '${KEY_ID}' found in keychain."
2148   fi
2149 done
2150 
2151 # check gpg encr secret encryption keys availability and fail
2152 # if none is available after a round of importing trials
2153 gpg_key_decryptable || \
2154 {
2155   echo "Missing secret keys for decryption in keychain."
2156   for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2157     KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2158     # test availability, try to import, retest
2159     if ! gpg_sec_avail "${KEY_ID}"; then
2160     echo "Try to import secret key '${KEY_ID}' from profile."
2161       gpg_import "${KEY_ID}" SEC
2162       gpg_key_cache RESET "${KEY_ID}"
2163     fi
2164   done
2165   gpg_key_decryptable || \
2166   {
2167     gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2168 has a secret key in the keychain. Decryption will be impossible!"
2169   }
2170 }
2171 
2172 # gpg secret sign key availability
2173 # if none set, autoset first encryption key as sign key
2174 if ! gpg_signing; then
2175   echo "Signing disabled per configuration."
2176 # try first key, if one set
2177 elif ! var_isset 'GPG_KEY_SIGN'; then
2178   KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2179   if [ -z "${KEY_ID}" ]; then
2180     echo "Signing disabled. No GPG_KEY entries in config."
2181     GPG_KEY_SIGN='disabled'
2182   else  
2183     # use avail OR try import OR fail
2184     if gpg_sec_avail "${KEY_ID}"; then
2185       GPG_KEY_SIGN="${KEY_ID}"
2186     else
2187       echo "Signing secret key '${KEY_ID}' not found."
2188       gpg_import "${KEY_ID}" SEC
2189       gpg_key_cache RESET "${KEY_ID}"
2190       if gpg_sec_avail "${KEY_ID}"; then
2191         GPG_KEY_SIGN="${KEY_ID}"
2192       fi
2193     fi
2194 
2195     # interpret sign key setting
2196     if var_isset 'GPG_KEY_SIGN'; then
2197       echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2198     else
2199       echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2200       GPG_KEY_SIGN='disabled'
2201     fi
2202   fi
2203 else
2204   KEY_ID="${GPG_KEY_SIGN}"
2205   if ! gpg_sec_avail "${KEY_ID}"; then
2206     inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2207     gpg_import "${KEY_ID}" SEC
2208     gpg_key_cache RESET "${KEY_ID}"
2209     gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2210   fi
2211 fi
2212 
2213 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2214 # key enc can deal without, but might profit from gpg-agent
2215 # if GPG_PW is not set alltogether
2216 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2217 if ! gpg_symmetric && \
2218    ( ! var_isset GPG_PW || \
2219      ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2220 
2221   if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2222     warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2223   elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2224     warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2225   elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2226     warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2227   else
2228     echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2229     GPG_USEAGENT="--use-agent"
2230   fi
2231 fi
2232 
2233 # end GPG config plausibility check2 
2234 fi
2235 
2236 # config plausibility check - SPACE ###########################################
2237 
2238 # is tmp is a folder and writable
2239 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2240 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2241 if [ "$?" != "0" ]; then
2242     error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2243 fi
2244 
2245 
2246 # get volsize, default duplicity volume size is 25MB since v0.5.07
2247 VOLSIZE=${VOLSIZE:-25}
2248 # double if asynch is on
2249 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2250 
2251 # TODO: check for enough (async= upload space and WARN only
2252 #       use function tmp_space 
2253 #echo TODO: reimplent tmp space check
2254 
2255 
2256 # test - GPG SANITY #####################################################################
2257 # if encryption is disabled, skip this whole section
2258 if gpg_disabled; then
2259   echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2260 elif ! gpg_testing; then 
2261   echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2262 else
2263 
2264 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2265 function cleanup_gpgtest { 
2266   echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2267   rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2268 }
2269 
2270 # signing enabled?
2271 if gpg_signing; then
2272   CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2273   CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2274 fi
2275 
2276 # using keys
2277 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2278 
2279   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2280     CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2281   done
2282   # check encrypting
2283   CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2284   run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2285   CMD_ERR=$?
2286 
2287   if [ "$CMD_ERR" != "0" ]; then 
2288     KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2289     [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2290   'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2291   can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2292     error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2293   fi
2294 
2295   # check decrypting
2296   CMD_MSG="Test - Decrypt"
2297   gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2298   run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2299   CMD_ERR=$?
2300 
2301   if [ "$CMD_ERR" != "0" ]; then 
2302     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2303   fi
2304 
2305 # symmetric only
2306 else
2307   # check encrypting
2308   CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2309   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2310   CMD_ERR=$?
2311   if [ "$CMD_ERR" != "0" ]; then 
2312     error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2313   fi
2314 
2315   # check decrypting
2316   CMD_MSG="Test - Decryption with passphrase"
2317   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2318   CMD_ERR=$?
2319   if [ "$CMD_ERR" != "0" ]; then 
2320     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2321   fi
2322 fi
2323 
2324 # compare original w/ decryptginal
2325 CMD_MSG="Test - Compare"
2326 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2327 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2328 CMD_ERR=$?
2329 if [ "$CMD_ERR" = "0" ]; then 
2330   cleanup_gpgtest
2331 else
2332   error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2333 fi
2334 
2335 fi # end disabled
2336 
2337 ## an empty line
2338 #echo
2339 
2340 # Exclude file is needed, create it if necessary
2341 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2342 
2343 # export only used keys, if bkp not already exists ######################################
2344 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2345 
2346 
2347 # command execution #####################################################################
2348 
2349 # urldecode url vars into plain text
2350 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2351 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2352 
2353 # defined TARGET_USER&PASS vars replace their URL pendants 
2354 # (double defs already dealt with)
2355 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2356 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2357 
2358 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2359 
2360 # issue some warnings
2361 case "$TARGET_URL_PROT_lowercase" in
2362   'cf+http')
2363     # info on missing AUTH_URL
2364     if ! var_isset 'CLOUDFILES_AUTHURL'; then
2365       inform "No CLOUDFILES_AUTHURL exported (in conf).
2366 Will use default which is probably rackspace."
2367     fi
2368     ;;
2369    'swift')
2370     # info on possibly missing AUTH_URL
2371        var_isset 'SWIFT_AUTHURL' ||\
2372        warning "\
2373 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2374     ;;
2375   'rsync')
2376     # everything in url (this backend does not support pass in env var)
2377     # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2378     # print warning older version is detected
2379     duplicity_version_lt 610 &&
2380       warning "\
2381 Duplicity version '$DUPL_VERSION' does not support providing the password as 
2382 env var for rsync backend. For security reasons you should consider to 
2383 update to a version greater than '0.6.10' of duplicity."
2384     ;;
2385 esac
2386 
2387 
2388 # for all protocols we put username in url and pass into env var 
2389 # for sec�rity reasons, we url_encode username to protect special chars
2390 # first sortout backends with special ways to handle password
2391 case "$TARGET_URL_PROT_lowercase" in
2392   'imap'|'imaps')
2393     var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2394     ;;
2395   *)
2396     # rest uses FTP_PASS var
2397     var_isset 'TARGET_URL_PASS' && \
2398       BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2399     ;;
2400 esac
2401 # insert url encoded username into target url if needed
2402 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2403   BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2404 else
2405   BACKEND_URL="$TARGET"
2406 fi
2407 
2408 
2409 # protect eval from special chars in url (e.g. open ')' in password, 
2410 # spaces in path, quotes) happens above in duplify() via quotewrap()
2411 SOURCE="$SOURCE"
2412 BACKEND_URL="$BACKEND_URL"
2413 EXCLUDE="$EXCLUDE"
2414 # since 0.7.03 --exclude-globbing-filelist is deprecated
2415 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist" 
2416 
2417 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2418 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2419   gsub(/\+/,\"_and_\",cmds);\
2420   gsub(/\-/,\"_or_\",cmds);\
2421   gsub(/\[/,\"_groupIn_\",cmds);\
2422   gsub(/\]/,\"_groupOut_\",cmds);\
2423   print cmds}")
2424 
2425 # split commands by '_', preserve spaces even if not allowed :)
2426 IFS='_' read -ra CMDS_IN <<< "$(tolower $cmds)"
2427 
2428 # convert cmds to array, 
2429 # post process, translate batch commands
2430 # ATTENTION: commands are lowercase from here on out
2431 declare -a CMDS
2432 for cmd in "${CMDS_IN[@]}"; do
2433   case "$cmd" in
2434     # backup -> [pre_bkp_post]
2435     'backup')
2436       CMDS=("${CMDS[@]}" groupin pre bkp post groupout)
2437     ;;
2438     # purgeAuto -> [purge purgeFull purgeIncr] depending on set conf vars 
2439     'purgeauto')
2440       purgeAuto=${MAX_AGE:+ purge}${MAX_FULL_BACKUPS:+ purgefull}${MAX_FULLS_WITH_INCRS:+ purgeincr}
2441       [[ -z "$purgeAuto" ]] && error "Command 'fullAuto' was given but neither of the purge conf vars configured."
2442       CMDS=("${CMDS[@]}" groupin $purgeAuto groupout)
2443     ;;
2444     *)
2445       CMDS=("${CMDS[@]}" "$cmd")
2446     ;;
2447   esac
2448 done
2449 #echo $(IFS=',';echo "${CMDS[*]}")
2450 
2451 unset FTPL_ERR
2452 
2453 # run CMDS
2454 for cmd in "${CMDS[@]}";
2455 do
2456 
2457 ## init
2458 # raise index in cmd array for pre/post param
2459 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2460 
2461 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2462 
2463 # get next cmd,cnd vars
2464 nextno=$(( $CMD_NO ))
2465 while ! var_isset 'CMD_NEXT'
2466 do
2467   nextno=$(($nextno+1))
2468   if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2469     CMD_VALUE=${CMDS[$nextno]}
2470     is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2471     is_groupMarker "$CMD_VALUE" && continue
2472     CMD_NEXT="$CMD_VALUE"
2473   else
2474     CMD_NEXT='END'
2475   fi
2476 done
2477 
2478 # get prev cnd, cnds are skipped pseudocmds
2479 prevno=$(( $CMD_NO ));
2480 while ! var_isset 'CND_PREV'
2481 do
2482   prevno=$(($prevno-1))
2483   if [ "$prevno" -ge 0 ]; then
2484     CMD_VALUE=${CMDS[$prevno]}
2485     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2486     is_command "$CMD_VALUE" && break
2487   else
2488     break
2489   fi
2490 done
2491 
2492 # get prev cmd command minus skipped commands, only executed
2493 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2494 while ! var_isset 'CMD_PREV'
2495 do
2496   prevno=$(($prevno-1))
2497   if [ "$prevno" -ge 0 ]; then
2498     CMD_VALUE=${CMDS[$prevno]}
2499     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2500     is_groupMarker "$CMD_VALUE" && continue
2501     CMD_PREV="$CMD_VALUE"
2502   else
2503     CMD_PREV='START'
2504   fi
2505 done
2506 
2507 function get_cmd_skip_count {
2508   # find closing bracket, get group skip count
2509   local nextno=$CMD_NO
2510   local GRP_OPEN=0
2511   local GRP_SKIP=0
2512   local CMD_VALUE
2513   while [ "$nextno" -lt "${#CMDS[@]}" ]
2514   do
2515     nextno=$(($nextno+1))
2516     CMD_VALUE=${CMDS[$nextno]}
2517     GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2518     if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2519         break;
2520     elif [ "$CMD_VALUE" == 'groupin' ]; then
2521       GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2522     elif [ "$CMD_VALUE" == 'groupout' ]; then
2523       GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2524       if [ "$GRP_OPEN" -lt 1 ]; then
2525         break;
2526       fi
2527     fi
2528   done
2529 
2530   echo $GRP_SKIP;
2531 }
2532 
2533 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2534 unset SKIP_NOW
2535 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2536   # skip cnd/grp cmds silently
2537   is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2538   CMD_SKIP=$(($CMD_SKIP - 1))
2539   SKIP_NOW="yes"
2540 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2541   CMD_SKIP=$(get_cmd_skip_count)
2542   # incl. this "cmd"
2543   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2544   unset CMD_SKIPPED
2545   SKIP_NOW="yes"
2546 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2547   CMD_SKIP=$(get_cmd_skip_count)
2548   # incl. this "cmd"
2549   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2550   unset CMD_SKIPPED
2551   SKIP_NOW="yes"
2552 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2553   unset 'CMD_SKIP';
2554   SKIP_NOW="yes"
2555 fi
2556 
2557 # let's do the skip now
2558 if [ -n "$SKIP_NOW" ]; then
2559   # sum up how many commands we actually skipped for the prev var routines
2560   CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2561   continue
2562 fi
2563 
2564 # save start time
2565 RUN_START=$(nsecs)
2566 
2567 # export some useful env vars for external scripts/programs to use
2568 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2569        TARGET_URL_USER TARGET_URL_PASS \
2570        GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2571        GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2572        RUN_START
2573 
2574 # user info
2575 echo; separator "Start running command $(toupper $cmd) at $(datefull_from_nsecs $RUN_START)"
2576 
2577 case "$(tolower $cmd)" in
2578   'pre'|'post')
2579     if [ "$cmd" == 'pre' ]; then
2580       script=$PRE
2581     else
2582       script=$POST
2583     fi
2584     # script execution in a subshell, protect us from failures/var overwrites
2585     ( run_script "$script" )
2586     ;;
2587   'bkp')
2588     duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2589           "$SOURCE" "$BACKEND_URL"
2590     ;;
2591   'incr')
2592     duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2593           "$SOURCE" "$BACKEND_URL"
2594     ;;
2595   'full')
2596     duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2597           "$SOURCE" "$BACKEND_URL"
2598     ;;
2599   'verify')
2600     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2601     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2602           "$BACKEND_URL" "$SOURCE"
2603     ;;
2604   'verifypath')
2605     TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2606     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2607     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2608   
2609   Hint: 
2610     Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2611 
2612     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2613           --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2614     ;;
2615   'list')
2616     # time param exists since 0.5.10+
2617     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2618     duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2619     ;;
2620   'cleanup')
2621     duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2622     ;;
2623   'purge')
2624     MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2625     [ -z "$MAX_AGE" ] && error "  Missing parameter <max_age>. Can be set in profile or as command line parameter."
2626     
2627     duplify remove-older-than "${MAX_AGE}" \
2628           -- "${dupl_opts[@]}" "$BACKEND_URL"
2629     ;;
2630   'purgefull')
2631     MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2632     [ -z "$MAX_FULL_BACKUPS" ] && error "  Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2633   
2634     duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2635           -- "${dupl_opts[@]}" "$BACKEND_URL"
2636     ;;
2637   'purgeincr')
2638     MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2639     [ -z "$MAX_FULLS_WITH_INCRS" ] && error "  Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2640   
2641     duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2642           -- "${dupl_opts[@]}" "$BACKEND_URL"
2643     ;;
2644   'restore')
2645     OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2646     [ -z "$OUT_PATH" ] && error "  Missing parameter target_path for restore.
2647   
2648   Hint: 
2649     Syntax is -> $ME <profile> restore <target_path> [<age>]"
2650     
2651     duplify  -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2652     ;;
2653   'fetch')
2654     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2655     TIME="${ftpl_pars[2]:-now}";
2656     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <src_path> or <target_path> for fetch.
2657   
2658   Hint: 
2659     Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2660     
2661     # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2662     duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2663               --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2664     ;;
2665   'status')
2666     duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2667     ;;    
2668   *)
2669     error "  Unknown command '$cmd'.
2670 
2671   Hint:
2672     Use '$ME usage' to get usage help."
2673     ;;
2674 esac
2675 
2676 CMD_ERR=$?
2677 RUN_END=$(nsecs)
2678 RUNTIME=$(( $RUN_END - $RUN_START ))
2679 
2680 # print message on error; set error code
2681 if [ "$CMD_ERR" -ne 0 ]; then
2682   error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2683   FTPL_ERR=1
2684 fi
2685 
2686 separator "Finished state $(error_to_string $CMD_ERR) at $(datefull_from_nsecs $RUN_END) - \
2687 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2688 
2689 done
2690 
2691 exit ${FTPL_ERR}