Code

From duply (simple duplicity) - a duplicity shell frontend
Jump to navigation Jump to search

Include Test

ERROR in secure-include.php::ef_include_get_errors (line 976) - Executing this 'php' code is currently prohibited because $wg_include_allowed_checksums[php] does not contain a matching checksum!

duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/

SVN access

svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply

SVN Log

https://sourceforge.net/p/ftplicity/code/commit_browser

Latest Development Snapshot

mod time

ERROR in secure-include.php::ef_include_get_errors (line 976) - Executing this 'shell' code is currently prohibited because $wg_include_allowed_checksums[shell] does not contain a matching checksum!

plain/text -> [/tmp/duply.sh]

   1 #!/usr/bin/env bash
   2 #
   3 ################################################################################
   4 #  duply (grown out of ftplicity), is a shell front end to duplicity that      #
   5 #  simplifies the usage by managing settings for backup jobs in profiles.      #
   6 #  It supports executing multiple commands in a batch mode to enable single    #
   7 #  line cron entries and executes pre/post backup scripts.                     #
   8 #  Since version 1.5.0 all duplicity backends are supported. Hence the name    #
   9 #  changed from ftplicity to duply.                                            #
  10 #  See http://duply.net or http://ftplicity.sourceforge.net/ for more info.    #
  11 #  (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany            #
  12 #  (c) 2008-2020 Edgar Soldin (changes since version 1.3)                      #
  13 ################################################################################
  14 #  LICENSE:                                                                    #
  15 #  This program is licensed under GPLv2.                                       #
  16 #  Please read the accompanying license information in gpl-2.0.txt.            #
  17 ################################################################################
  18 #  TODO/IDEAS/KNOWN PROBLEMS:
  19 #  - possibility to restore time frames (incl. deleted files)
  20 #    realizable by listing each backup and restore from 
  21 #    oldest to the newest, problem: not performant
  22 #  - search file in all backups function and show available
  23 #    versions with backups date (list old avail since 0.6.06)
  24 #  - edit profile opens conf file in vi 
  25 #  - implement log-fd interpretation
  26 #  - add a duplicity option check against the options pending 
  27 #    deprecation since 0.5.10 namely --time-separator
  28 #                               --short-filenames
  29 #                              --old-filenames
  30 #  - add 'exclude_<command>' list usage e.g. exclude_verify
  31 #  - featreq 25: a download/install duplicity option
  32 #  - hint on install software if a piece is missing
  33 #  - import/export profile from/to .tgz function !!!
  34 #
  35 #  CHANGELOG:
  36 #  2.3.1 (11.2.2021)
  37 #  - bugfix 123: symmetric encryption errs out, asks for '' private key
  38 #
  39 #  2.3 (30.12.2020)
  40 #  - don't import whole key pair anymore if only pub/sec is requested
  41 #  - gpg import routine informs on missing key files in profile now
  42 #  - add check/import needed secret key for decryption
  43 #  - featreq 50: Disable GPG key backups, implemented/added settings
  44 #      GPG_IMPORT/GPG_EXPORT='disabled' to conf template
  45 #
  46 #  2.2.2 (24.02.2020)
  47 #  - bugfix 120: Failures in "Autoset trust of key" during restore 
  48 #    because of gpg2.2 fingerprint output change
  49 #
  50 #  2.2.1 (22.01.2020)
  51 #  - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  52 #  - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  53 #  - fix abbreviation spelling of 'e.g.'
  54 #
  55 #  2.2 (30.12.2018)
  56 #  - featreq 44: implement grouping for batch commands
  57 #      new separators are [] (square brackets) or groupIn/groupOut
  58 #      command 'backup' translates now to [pre_bkp_post] to be skipped as
  59 #      one block in case a condition was set in the batch instruction
  60 #
  61 #  2.1 (23.07.2018)
  62 #  - be more verbose when duplicity version detection fails
  63 #  - using info shows python binary's path for easier identification now
  64 #  - reworked python interpreter handling, it's either
  65 #      configured per PYTHON var
  66 #      unconfigured, parsed from duplicity shebang
  67 #      or set to current duplicity default 'python2' (was 'python' until now)
  68 #  - do not quotewrap strings because of slashes (e.g. paths) anymore
  69 #  - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  70 #
  71 #  2.0.4 (20.02.2018)
  72 #  - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  73 #  - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  74 #  - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  75 #  - bugfix debian 882159: duply: occasionally shows negative runtimes
  76 #
  77 #  2.0.3 (29.08.2017)
  78 #  - bugfix: "line 2231: CMDS: bad array subscript"
  79 #  - bugfix 112: "env: illegal option -- u" on MacOSX
  80 #
  81 #  2.0.2 (23.05.2017)
  82 #  - bugfix: never insert creds into file:// targets
  83 #  - bugfix: avail profiles hint sometimes shortend the names by one char
  84 #  - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  85 #  - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  86 #  - bugfix 97: Unknown command should be ERROR, not WARNING 
  87 #
  88 #  2.0.1 (16.11.2016)
  89 #  - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
  90 #
  91 #  2.0 (27.10.2016)
  92 #  made this a major version change, as we broke backward compatibility anyway
  93 #  (see last change in v1.10). got complaints that rightfully pointed out
  94 #  that should only come w/ a major version change. so, here we go ;)
  95 #  if your backend stops working w/ this version create a new profile and
  96 #  export the env vars needed as described in the comments of the conf file
  97 #  directly above the SOURCE setting.
  98 #  Changes:
  99 #  - making sure multi spaces in TARGET survive awk processing
 100 #  - new env var PROFILE exported to scripts 
 101 #  - fix 102: expose a unique timestamp variable for pre/post scripts
 102 #    actually a featreq. exporting RUN_START nanosec unix timestamp
 103 #  - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
 104 #  - fix 96: duply cannot handle two consecutive spaces in paths
 105 #
 106 #  1.11.3 (29.5.2016)
 107 #  - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
 108 #
 109 #  1.11.2 (11.2.2016)
 110 #  - fix "gpg: unsafe" version print out 
 111 #  - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
 112 #  - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional 
 113 #    documentation about needed env vars to template conf file
 114 #
 115 #  1.11.1 (18.12.2015)
 116 #  - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
 117 #  - reverted duply's default PYTHON to 'python'
 118 #
 119 #  1.11 (24.11.2015)
 120 #  - remove obsolete --ssh-askpass routine
 121 #  - add PYTHON conf var to allow global override of used python interpreter
 122 #  - enforced usage of "python2" in PATH as default interpreter for internal
 123 #    use _and_ to run duplicity (setup.py changed the shebang to the fixed
 124 #    path /usr/bin/python until 0.7.05, which we circumvent this way)
 125 #  - featreq 36: support gpg-connect-agent as a means to detect if an agent is 
 126 #    running (thx Thomas Harning Jr.), used gpg-agent for detection though
 127 #  - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
 128 #  - key export routine respects gpg-agent usage now
 129 #
 130 #  1.10.1 (19.8.2015)
 131 #  - bugfix 86: Duply+Swift outputs warning
 132 #  - bugfix 87: Swift fails without BACKEND_URL
 133 #
 134 #  1.10 (31.7.2015)
 135 #  - featreq 37: busybox issues - fix awk, grep version detection,
 136 #    fix grep failure because --color=never switch is unsupported
 137 #    (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
 138 #  - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
 139 #    (thx Joachim Wiedorn, also for maintaining the debian package)
 140 #  - implemented base-/dirname as bash functions
 141 #  - featreq 31 " Support for duplicity Azure backend " - ignored a 
 142 #    contributed patch by Scott McKenzie and instead opted for removing almost
 143 #    all code that deals with special env vars required by backends.
 144 #    adding and modifying these results in too much overhead so i dropped this
 145 #    feature. the future alternative for users is to consult the duplicity 
 146 #    manpage and add the needed export definitions to the conf file.
 147 #    appended a commented example to the template conf below the auth section.
 148 #
 149 #  1.9.2 (21.6.2015)
 150 #  - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
 151 #  - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
 152 #  - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
 153 #  - added GPG conf var (see conf template for details)
 154 #  - added grep version output as it is an integral needed binary
 155 #  - added PYTHONPATH printout in version output
 156 #
 157 #  1.9.1 (13.10.2014)
 158 #  - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
 159 #  - bugfix: CMD_PREV contained command even if it was skipped
 160 #
 161 #  1.9.0 (24.8.2014)
 162 #  - bugfix: env vars were not exported when external script was executable
 163 #  - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) 
 164 #    see gpg manpage, section "How to specify a user ID"
 165 #    let gpg complain when the delivered values are invalid for whatever reason
 166 #  - started to rework tmp space checking, exposed folder & writable check
 167 #    TODO: reimplement enough file space available checking
 168 #
 169 #  1.8.0 (13.7.2014)
 170 #  - add command verifyPath to expose 'verify --file-to-restore' action
 171 #  - add time parameter support to verify command
 172 #  - add section time formats to usage output 
 173 #
 174 #  1.7.4 (24.6.2014)
 175 #  - remove ubuntu one support, service is discontinued
 176 #  - featreq 31: add authenticated swift (contributed by Justus Seifert)
 177 #
 178 #  1.7.3 (3.4.2014)
 179 #  - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
 180 #
 181 #  1.7.2 (1.4.2014 "April,April")
 182 #  - bugfix: debian Bug#743190 "duply no longer allows restoration without 
 183 #     gpg passphrase in conf file"
 184 #     GPG_AGENT_INFO env var is now needed to trigger --use-agent
 185 #  - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
 186 #
 187 #  1.7.1 (30.3.2014)
 188 #  - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to 
 189 #     incompatibility with new minus batch separator 
 190 #
 191 #  1.7.0 (20.3.2014)
 192 #  - disabled gpg key id plausibility check, too many valid possibilities
 193 #  - featreq 7 "Halt if precondition fails":
 194 #     added and(+), or(-) batch command(separator) support
 195 #  - featreq 26 "pre/post script with shebang line": 
 196 #     if a script is flagged executable it's executed in a subshell 
 197 #     now as opposed to sourced to bash, which is the default
 198 #  - bugfix: do not check if dpbx, swift credentials are set anymore 
 199 #  - bugfix: properly escape profile name, archdir if used as arguments
 200 #  - add DUPL_PRECMD conf setting for use with e.g. trickle
 201 #
 202 #  1.6.0 (1.1.2014)
 203 #  - support gs backend
 204 #  - support dropbox backend
 205 #  - add gpg-agent support to gpg test routines
 206 #  - autoenable --use-agent if passwords were not defined in config
 207 #  - GPG_OPTS are now honored everywhere, keyrings or complete gpg
 208 #    homedir can thus be configured to be located anywhere
 209 #  - always import both secret and public key if avail from config profile
 210 #  - new explanatory comments in initial exclude file
 211 #  - bugfix 7: Duply only imports one key at a time 
 212 #
 213 #  1.5.11 (19.07.2013)
 214 #  - purge-incr command for remove-all-inc-of-but-n-full feature added
 215 #    patch provided by Moritz Augsburger, thanks!
 216 #  - documented version command in man page
 217 #
 218 #  1.5.10 (26.03.2013)
 219 #  - minor indent and documentation fixes
 220 #  - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
 221 #  - bugfix: fix url_decoding generally and for python3
 222 #  - bugfix 3609075: wrong script results in status line (thx David Epping)
 223 #
 224 #  1.5.9 (22.11.2012)
 225 #  - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
 226 #  - restore/fetch now also ignores --include* or --exclude='foobar' 
 227 #
 228 #  1.5.8 (26.10.2012)
 229 #  - bugfix 3575487: implement proper cloud files support
 230 #
 231 #  1.5.7 (10.06.2012)
 232 #  - bugfix 3531450: Cannot use space in target URL (file:///) anymore
 233 #
 234 #  1.5.6 (24.5.2012)
 235 #  - commands purge, purge-full have no default value anymore for security 
 236 #    reasons; instead max value can be given via cmd line or must be set
 237 #    in profile; else an error is shown.
 238 #  - minor man page modifications
 239 #
 240 #  versioning scheme will be simplified to [major].[minor].[patch] version
 241 #  with the next version raise
 242 #
 243 #  1.5.5.5 (4.2.2012)
 244 #  - bugfix 3479605: SEL context confused profile folder's permission check
 245 #  - colon ':' in url passphrase got ignored, added python driven url_decoding
 246 #    for user & pass to better process special chars
 247 #
 248 #  1.5.5.4 (16.10.2011)
 249 #  - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
 250 #  - add support for separate sign passphrase (needs duplicity 0.6.14+)
 251 #
 252 #  1.5.5.3 (1.10.2011)
 253 #  - bugfix 3416690: preview threw echo1 error
 254 #  - fix unknown cmds error usage & friends if more than 2 params were given
 255 #
 256 #  1.5.5.2 (23.9.2011)
 257 #  - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
 258 #  - bugfix: mawk does not support \W and did not split multikey definitions
 259 #  - all parameters should survive  single (') and double (") quotes now
 260 #
 261 #  1.5.5.1 (7.6.2011)
 262 #  - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
 263 #  - bugfix 3312208: signing detection broke symmetric gpg test routine
 264 #
 265 #  1.5.5 (2.5.2011)
 266 #  - bugfix: fetch problem with space char in path, escape all params 
 267 #    containing non word chars
 268 #  - list available profiles, if given profile cannot be found
 269 #  - added --use-agent configuration hint
 270 #  - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke 
 271 #    fetch/restore
 272 #  - version command now prints out 'using installed' info
 273 #  - featreq 3166169: autotrust imported keys, based on code submitted by 
 274 #    Martin Ellis - imported keys are now automagically trusted ultimately 
 275 #  - new txt2man feature to create manpages for package maintainers
 276 #
 277 #  1.5.4.2 (6.1.2011)
 278 #  - new command changelog
 279 #  - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
 280 #  - bugfix: freebsd awk hangs on 'awk -W version' 
 281 #  - bugfix 3150244: mawk does not know '--version'
 282 #  - minor help text improvements
 283 #  - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
 284 #
 285 #  1.5.4.1 (4.12.2010)
 286 #  - output awk, python, bash version now in prolog
 287 #  - shebang uses /usr/bin/env now for freebsd compatibility, 
 288 #    bash not in /bin/bash 
 289 #  - new --disable-encryption parameter, 
 290 #    to override profile encr settings for one run
 291 #  - added exclude-if-present setting to conf template
 292 #  - bug 3126972: GPG_PW only needed for signing/symmetric encryption 
 293 #    (even though duplicity still needs it)
 294 #
 295 #  1.5.4 (15.11.2010)
 296 #  - as of 1.5.3 already, new ARCH_DIR config option
 297 #  - multiple key support
 298 #  - ftplicity-Feature Requests-2994929: separate encryption and signing key
 299 #  - key signing of symmetric encryption possible (duplicity patch committed)
 300 #  - gpg tests disable switch
 301 #  - gpg tests now previewable and more intelligent
 302 #
 303 #  1.5.3 (1.11.2010)
 304 #  - bugfix 3056628: improve busybox compatibility, grep did not have -m param
 305 #  - bugfix 2995408: allow empty password for PGP key
 306 #  - bugfix 2996459: Duply erroneously escapes '-' symbol in username
 307 #  - url_encode function is now pythonized
 308 #  - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
 309 #  - feature 3059262: Make pre and post aware of parameters, 
 310 #                     internal parameters + CMD of pre or post 
 311 #
 312 #  1.5.2.3 (16.4.2010)
 313 #  - bugfix: date again, should now work virtually anywhere
 314 #
 315 #  1.5.2.2 (3.4.2010)
 316 #  - minor bugfix: duplicity 0.6.8b version string now parsable
 317 #  - added INSTALL.txt
 318 #
 319 #  1.5.2.1 (23.3.2010)
 320 #  - bugfix: date formatting is awked now and should work on all platforms
 321 #
 322 #  1.5.2 (2.3.2010)
 323 #  - bugfix: errors print to STD_ERR now, failed tasks print an error message
 324 #  - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
 325 #  - simplified & cleaned profileless commands, removed second instance
 326 #  - generalized separator time routines
 327 #  - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
 328 #  - minor fixes
 329 #
 330 #  1.5.1.5 (5.2.2010)
 331 #  - bugfix: added special handling of credentials for rsync, imap(s)
 332 #
 333 #  1.5.1.4 (7.1.2010)
 334 #  - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
 335 #  - check if ncftp binary is available if url protocol is ftp
 336 #  - bugfix: duplicity output is now printed to screen directly to resolve
 337 #            'mem alloc problem' bug report
 338 #  - bugfix: passwords will not be in the url anymore to solve the 'duply shows
 339 #            sensitive data in process listing' bug report
 340 #
 341 #  1.5.1.3 (24.12.2009) 'merry xmas'
 342 #  - bugfix: gpg pass now apostrophed to allow space and friends
 343 #  - bugfix: credentials are now url encoded to allow special chars in them
 344 #            a note about url encoding has been added to the conf template
 345 #
 346 #  1.5.1.2 (1.11.2009)
 347 #  - bugfix: open parenthesis in password broke duplicity execution
 348 #  - bugfix: ssh/scp backend does not always need credentials e.g. key auth
 349 #
 350 #  1.5.1.1 (21.09.2009)
 351 #  - bugfix: fixed s3[+http] TARGET_PASS not needed routine
 352 #  - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
 353 #    see https://sourceforge.net/tracker/index.php?func=detail&
 354 #                aid=2864410&group_id=217745&atid=1041147
 355 #
 356 #  1.5.1 (21.09.2009) - duply (fka. ftplicity)
 357 #  - first things first: ftplicity (being able to support all backends since 
 358 #    some time) will be called duply (fka. ftplicity) from now on. The addendum
 359 #    is for the time being to circumvent confusion.
 360 #  - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity 
 361 #            command failed
 362 #  - s3[+http] now supported natively by translating user/pass to access_key/
 363 #    secret_key environment variables needed by duplicity s3 boto backend 
 364 #  - bugfix: additional output lines do not confuse version check anymore
 365 #  - list command supports now age parameter (patch by stefan on feature 
 366 #    request tracker)
 367 #  - bugfix: option/param pairs are now correctly passed on to duplicity
 368 #  - bugfix: s3[+http] needs no TARGET_PASS if command is read only
 369 #
 370 #  1.5.0.2 (31.07.1009)
 371 #  - bugfix: insert password in target url didn't work with debian mawk
 372 #            related to previous bug report
 373 #
 374 #  1.5.0.1 (23.07.2009)
 375 #  - bugfix: gawk gensub dependency raised an error on debian's default mawk
 376 #            replaced with match/substr command combination (bug report)
 377 #            https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
 378 #            group_id=217745
 379 #
 380 #  1.5.0 (01.07.2009)
 381 #  - removed ftp limitation, all duplicity backends should work now
 382 #  - bugfix: date for separator failed on openwrt busybox date, added a 
 383 #    detecting workaround, milliseconds are not available w/ busybox date
 384 #
 385 #  1.4.2.1 (14.05.2009)
 386 #  - bugfix: free temp space detection failed with lvm, fixed awk parse routine
 387 #
 388 #  1.4.2 (22.04.2009)
 389 #  - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
 390 #    armored ascii nature, the id helps if the key is switched for some reason
 391 #    im/export routines are updated accordingly (import is backward compatible 
 392 #    to the old profile/gpgkey files)         
 393 #  - profile argument is treated as path if it contains slashes 
 394 #    (for details see usage)
 395 #  - non-ftplicity options (all but --preview currently) are now passed 
 396 #    on to duplicity 
 397 #  - removed need for stat in secure_conf, it is ls based now
 398 #  - added profile folder readable check
 399 #  - added gpg version & home info output
 400 #  - awk utility availability is now checked, because it was mandatory already
 401 #  - tmp space is now checked on writability and space requirement
 402 #    test fails on less than 25MB or configured $VOLSIZE, 
 403 #    test warns if there is less than two times $VOLSIZE because 
 404 #    that's required for --asynchronous-upload option  
 405 #  - gpg functionality is tested now before executing duplicity 
 406 #    test drive contains encryption, decryption, comparison, cleanup
 407 #    this is meant to detect non trusted or other gpg errors early
 408 #  - added possibility of doing symmetric encryption with duplicity
 409 #    set GPG_KEY="" or simply comment it out
 410 #  - added hints in config template on the depreciation of 
 411 #    --short-filenames, --time-separator duplicity options
 412 #
 413 #  new versioning scheme 1.4.2b => 1.4.2, 
 414 #  beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
 415 #  to the first bug fixing version and 1.4.2.2 to the second and so on
 416 #  also the releases will now have a release date formatted (Day.Month.Year)
 417 #
 418 #  1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
 419 #            named exactly as the profile if existing in executing dir
 420 #          - improved plausibility checking of config and profile folder
 421 #          - secure_conf only acts if needed and prints a warning now
 422 #
 423 #  1.4.1b  - introduce status (duplicity collection-status) command
 424 #          - pre/post script output printed always now, not only on errors
 425 #          - new config parameter GPG_OPTS to pass gpg options
 426 #            added examples & comments to profile template conf
 427 #          - reworked separator times, added duration display
 428 #          - added --preview switch, to preview generated command lines
 429 #          - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
 430 #            profiles because they have reasonable defaults now if not set
 431 #
 432 #  1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
 433 #            therefore backup translates to pre_bkp_post now
 434 #          - bugfix: new command bkp, which represents duplicity's 
 435 #            default action (incr or full if full_if_older matches
 436 #            or no earlier backup chain is found)
 437 #
 438 #  new versioning scheme 1.4 => 1.4.0, added new minor revision number
 439 #  this is meant to slow down the rapid version growing but still keep 
 440 #  versions cleanly separated.
 441 #  only additional features will raise the new minor revision number. 
 442 #  all releases start as beta, each bugfix release will raise the beta 
 443 #  count, usually new features arrive before a version 'ripes' to stable
 444 #    
 445 #  1.4.0b
 446 #    1.4b  - added startup info on version, time, selected profile
 447 #          - added time output to separation lines
 448 #          - introduced: command purge-full implements duplicity's 
 449 #            remove-all-but-n-full functionality (patch by unknown),
 450 #            uses config variable $MAX_FULL_BACKUPS (default = 1)
 451 #          - purge config var $MAX_AGE defaults to 1M (month) now 
 452 #          - command full does not execute pre/post anymore
 453 #            use batch command pre_full_post if needed 
 454 #          - introduced batch mode cmd1_cmd2_etc
 455 #            (in turn removed the bvp command)
 456 #          - unknown/undefined command issues a warning/error now
 457 #          - bugfix: version check works with 0.4.2 and older now
 458 #    1.3b3 - introduced pre/post commands to execute/debug scripts
 459 #          - introduced bvp (backup, verify, purge)
 460 #          - bugfix: removed need for awk gensub, now mawk compatible
 461 #    1.3b2 - removed pre/post need executable bit set 
 462 #          - profiles now under ~/.ftplicity as folders
 463 #          - root can keep profiles in /etc/ftplicity, folder must be
 464 #            created by hand, existing profiles must be moved there
 465 #          - removed ftplicity in path requirement
 466 #          - bugfix: bash < v.3 did not know '=~'
 467 #          - bugfix: purge works again 
 468 #    1.3   - introduces multiple profiles support
 469 #          - modified some script errors/docs
 470 #          - reordered gpg key check import routine
 471 #          - added 'gpg key id not set' check
 472 #          - added error_gpg (adds how to setup gpg key howto)
 473 #          - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
 474 #          - duplicity_version_check routine introduced
 475 #          - added time separator, shortnames, volsize, full_if_older 
 476 #            duplicity options to config file (inspired by stevie 
 477 #            from http://weareroot.de) 
 478 #    1.1.1 - bugfix: encryption reactivated
 479 #    1.1   - introduced config directory
 480 #    1.0   - first release
 481 ################################################################################
 482 
 483 # utility functions overriding binaries
 484 
 485 # wrap grep to override possible env set GREP_OPTIONS=--color=always
 486 function grep {
 487   command env "GREP_OPTIONS=" grep "$@"
 488 }
 489 
 490 # implement basename in plain bash
 491 function basename {
 492   local stripped="${1%/}"
 493   echo "${stripped##*/}"
 494 }
 495 
 496 # implement dirname in plain bash
 497 function dirname {
 498   echo ${1%/*}
 499 }
 500 
 501 # implement basic which in plain bash
 502 function which {
 503   type -p "$@"
 504 }
 505 
 506 # check availability of executables via file name or file paths
 507 function lookup {
 508   local bin="$1"
 509   # look for file names in path via bash hash OR 
 510   # look for executables at given relative/absolute location
 511   ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
 512 }
 513 
 514 # the python binary to use, exit code 0 when configured, else 1
 515 function python_binary {
 516   # if unset, parse from duplicity shebang
 517   if ! var_isset 'PYTHON'; then
 518     duplicity_python_binary_parse;
 519     echo $DUPL_PYTHON_BIN;
 520     return 1;
 521   else
 522     # tell if PYTHON was configured manually
 523     echo $PYTHON;
 524     return 0
 525   fi
 526 }
 527 
 528 # important definitions #######################################################
 529 
 530 ME_LONG="$0"
 531 ME="$(basename $0)"
 532 ME_NAME="${ME%%.*}"
 533 ME_VERSION="2.3.2dev"
 534 ME_WEBSITE="http://duply.net"
 535 
 536 # default config values
 537 DEFAULT_SOURCE='/path/of/source'
 538 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
 539 DEFAULT_TARGET_USER='_backend_username_'
 540 DEFAULT_TARGET_PASS='_backend_password_'
 541 DEFAULT_GPG='gpg'
 542 DEFAULT_GPG_KEY='_KEY_ID_'
 543 DEFAULT_GPG_PW='_GPG_PASSWORD_'
 544 DEFAULT_PYTHON='python2'
 545 
 546 # function definitions ##########################
 547 
 548 function set_config { # sets global config vars
 549   local CONFHOME_COMPAT="$HOME/.ftplicity"
 550   local CONFHOME="$HOME/.duply"
 551   local CONFHOME_ETC_COMPAT="/etc/ftplicity"
 552   local CONFHOME_ETC="/etc/duply"
 553 
 554   # confdir can be delivered as path (must contain /)
 555   if [ `echo $FTPLCFG | grep /` ] ; then 
 556     CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
 557               ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
 558                && echo $(pwd)/${FTPLCFG} ) || \
 559               echo ${FTPLCFG})          
 560   # or DEFAULT in home/.duply folder (NEW)
 561   elif [ -d "${CONFHOME}" ]; then
 562     CONFDIR="${CONFHOME}/${FTPLCFG}"
 563   # or in home/.ftplicity folder (OLD)
 564   elif [ -d "${CONFHOME_COMPAT}" ]; then
 565     CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
 566     warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
 567   # root can put profiles under /etc/duply (NEW) if path exists
 568   elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
 569     CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
 570   # root can keep profiles under /etc/ftplicity (OLD) if path exists
 571   elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
 572     CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
 573     warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
 574   # hmm no profile folder there, then use default for error later
 575   else
 576     CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
 577   fi
 578 
 579   # remove trailing slash, get profile name etc.
 580   CONFDIR="${CONFDIR%/}"
 581   PROFILE="${CONFDIR##*/}"
 582   CONF="$CONFDIR/conf"
 583   PRE="$CONFDIR/pre"
 584   POST="$CONFDIR/post"
 585   EXCLUDE="$CONFDIR/exclude"
 586   KEYFILE="$CONFDIR/gpgkey.asc"
 587 }
 588 
 589 function version_info { # print version information
 590   cat <<END
 591   $ME_NAME version $ME_VERSION
 592   ($ME_WEBSITE)
 593 END
 594 }
 595 
 596 function version_info_using { 
 597   cat <<END
 598 $(version_info)
 599 
 600   $(using_info)
 601 END
 602 }
 603 
 604 function using_info {
 605   # init needed vars into global name space
 606   lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
 607   local NOTFOUND="INVALID"
 608   local AWK_VERSION GREP_VERSION PYTHON_RUNNER
 609   # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
 610   AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 611   GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 612   PYTHON_RUNNER=$(python_binary)
 613   local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
 614   local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
 615   local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
 616   # print out
 617   echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
 618 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
 619 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
 620 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
 621 }
 622 
 623 function usage_info { # print usage information
 624 
 625   cat <<USAGE_EOF
 626 VERSION:
 627 $(version_info)
 628   
 629 DESCRIPTION: 
 630   Duply deals as a wrapper for the mighty duplicity magic.
 631   It simplifies running duplicity with cron or on command line by:
 632 
 633     - keeping recurring settings in profiles per backup job
 634     - enabling batch operations e.g. backup_verify+purge
 635     - executing pre/post scripts (different actions possible 
 636       depending on previous or next command or it's exit status)
 637     - precondition checking for flawless duplicity operation
 638 
 639   For each backup job one configuration profile must be created.
 640   The profile folder will be stored under '~/.${ME_NAME}/<profile>'
 641   (where ~ is the current users home directory).
 642   Hint:  
 643    If the folder '/etc/${ME_NAME}' exists, the profiles for the super
 644    user root will be searched & created there.
 645 
 646 USAGE:
 647   first time usage (profile creation):  
 648     $ME <profile> create
 649 
 650   general usage in single or batch mode (see EXAMPLES):  
 651     $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
 652 
 653   For batches the conditional separators can also be written as pseudo commands
 654   and(+), or(-). See SEPARATORS for details.
 655 
 656   Non $ME options are passed on to duplicity (see OPTIONS).
 657   All conf parameters can also be defined in the environment instead.
 658 
 659 PROFILE:
 660   Indicated by a path or a profile name (<profile>), which is resolved 
 661   to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
 662 
 663   Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
 664   the folder manually before running $ME_NAME as superuser.
 665   Note:  
 666     Already existing profiles in root's home folder will cease to work
 667     unless they are moved to the new location manually.
 668 
 669   example 1:   $ME humbug backup
 670 
 671   Alternatively a _path_ might be used e.g. useful for quick testing, 
 672   restoring or exotic locations. Shell expansion should work as usual.
 673   Hint:  
 674     The path must contain at least one path separator '/', 
 675     e.g. './test' instead of only 'test'.
 676 
 677   example 2:   $ME ~/.${ME_NAME}/humbug backup
 678 
 679 SEPARATORS:
 680   _ (underscore)  
 681              neutral separator
 682   + (plus sign), _and_  
 683              conditional AND
 684              the next command will only be executed if the previous succeeded
 685   - (minus sign), _or_  
 686              conditional OR
 687              the next command will only be executed if the previous failed
 688   [] (square brackets), _groupIn_/_groupOut_  
 689              enables grouping of commands
 690 
 691    example:  
 692     'pre+[bkp-verify]_post' translates to
 693     'pre_and_groupIn_bkp_or_verify_groupOut_post'
 694 
 695 COMMANDS:
 696   usage      get usage help text
 697 
 698   and/or/groupIn/groupOut  
 699              pseudo commands used in batches (see SEPARATORS above)
 700 
 701   create     creates a configuration profile
 702   backup     backup with pre/post script execution (batch: [pre_bkp_post]),
 703               full (if full_if_older matches or no earlier backup is found)
 704               incremental (in all other cases)
 705   pre/post   execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
 706   bkp        as above but without executing pre/post scripts
 707   full       force full backup
 708   incr       force incremental backup
 709   list [<age>]  
 710              list all files in backup (as it was at <age>, default: now)
 711   status     prints backup sets and chains currently in repository
 712   verify [<age>] [--compare-data]  
 713              list files changed, since age if given
 714   verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]  
 715              list changes of a file or folder path in backup compared to a
 716              local path, since age if given
 717   restore <target_path> [<age>]  
 718              restore the complete backup to <target_path> [as it was at <age>]
 719   fetch <src_path> <target_path> [<age>]  
 720              fetch single file/folder from backup [as it was at <age>]
 721   purge [<max_age>] [--force]  
 722              list outdated backup files (older than \$MAX_AGE)
 723               [use --force to actually delete these files]
 724   purgeFull [<max_full_backups>] [--force]  
 725              list outdated backup files (\$MAX_FULL_BACKUPS being the number of
 726              full backups and associated incrementals to keep, counting in 
 727              reverse chronological order)
 728               [use --force to actually delete these files]
 729   purgeIncr [<max_fulls_with_incrs>] [--force]  
 730              list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being 
 731              the number of full backups which associated incrementals will be
 732              kept, counting in reverse chronological order) 
 733               [use --force to actually delete these files]
 734   cleanup [--force]  
 735              list broken backup chain files archives (e.g. after unfinished run)
 736               [use --force to actually delete these files]
 737 
 738   changelog  print changelog / todo list
 739   txt2man    feature for package maintainers - create a manpage based on the 
 740              usage output. download txt2man from http://mvertes.free.fr/, put 
 741              it in the PATH and run '$ME txt2man' to create a man page.
 742   version    show version information of $ME_NAME and needed programs
 743 
 744 OPTIONS:
 745   --force    passed to duplicity (see commands: 
 746              purge, purgeFull, purgeIncr, cleanup)
 747   --preview  do nothing but print out generated duplicity command lines
 748   --disable-encryption  
 749              disable encryption, overrides profile settings
 750 
 751 TIME FORMATS:
 752   For all time related parameters like age, max_age etc.
 753   Refer to the duplicity manpage for all available formats. Here some examples:
 754     2002-01-25T07:00:00+02:00 (full date time format string)
 755     2002/3/5 (date string YYYY/MM/DD)
 756     12D (interval, 12 days ago)
 757     1h78m (interval, 1 hour 78 minutes ago)
 758 
 759 PRE/POST SCRIPTS:
 760   Some useful internal duply variables are exported to the scripts.
 761 
 762     PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>, 
 763     GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
 764     CMD_<PREV|NEXT> (previous/next command), 
 765     CND_<PREV|NEXT> (condition before/after)
 766 
 767   The CMD_* variables were introduced to allow different actions according to 
 768   the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post' 
 769   will call the pre script two times, with CMD_NEXT variable set to 'bkp' 
 770   on the first and to 'verify' on the second run.
 771   CMD_ERR holds the exit code of the CMD_PREV .
 772 
 773 EXAMPLES:
 774   create profile 'humbug':  
 775     $ME humbug create (don't forget to edit this new conf file)
 776   backup 'humbug' now:  
 777     $ME humbug backup
 778   list available backup sets of profile 'humbug':  
 779     $ME humbug status
 780   list and delete outdated backups of 'humbug':  
 781     $ME humbug purge --force
 782   restore latest backup of 'humbug' to /mnt/restore:  
 783     $ME humbug restore /mnt/restore
 784   restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:  
 785     $ME humbug fetch etc/passwd /root/pw 4D
 786     (see "duplicity manpage", section TIME FORMATS)
 787   a one line batch job on 'humbug' for cron execution:  
 788     $ME humbug backup_verify_purge --force
 789   batch job to run a full backup with pre/post scripts:  
 790     $ME humbug pre_full_post
 791 
 792 FILES:
 793   in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
 794   conf             profile configuration file
 795   pre,post         pre/post scripts (see above for details)
 796   gpgkey.*.asc     exported GPG key files
 797   exclude          a globbing list of included or excluded files/folders
 798                    (see "duplicity manpage", section FILE SELECTION)
 799 
 800 $(hint_profile)
 801 
 802 SEE ALSO:
 803   duplicity man page:
 804     duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
 805 USAGE_EOF
 806 }
 807 
 808 # to check call 'duply txt2man | man -l -'
 809 function usage_txt2man {
 810   usage_info | \
 811   awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
 812   txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
 813 }
 814 
 815 function changelog {
 816   cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#(  )?/,"",$0);print}'
 817 }
 818 
 819 function create_config {
 820   if [ ! -d "$CONFDIR" ] ; then
 821     mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
 822   # create initial config file
 823     cat <<EOF >"$CONF"
 824 # gpg encryption settings, simple settings:
 825 #  GPG_KEY='disabled' - disables encryption alltogether
 826 #  GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
 827 #   sign if secret key of key1 is available use GPG_PW for sign & decrypt
 828 #  Note: you can specify keys via all methods described in gpg manpage,
 829 #        section "How to specify a user ID", escape commas (,) via backslash (\)
 830 #        e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
 831 #        as they are used to separate the entries
 832 #  GPG_PW='passphrase' - symmetric encryption using passphrase only
 833 GPG_KEY='${DEFAULT_GPG_KEY}'
 834 GPG_PW='${DEFAULT_GPG_PW}'
 835 # gpg encryption settings in detail (extended settings)
 836 #  the above settings translate to the following more specific settings
 837 #  GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
 838 #  GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
 839 #  GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
 840 #   encryption. If you want to deliver different passphrases for e.g. 
 841 #   several keys or symmetric encryption plus key signing you can use
 842 #   gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
 843 #   also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage 
 844 # notes on en/decryption
 845 #  private key and passphrase will only be needed for decryption or signing.
 846 #  decryption happens on restore and incrementals (compare archdir contents).
 847 #  for security reasons it makes sense to separate the signing key from the
 848 #  encryption keys. https://answers.launchpad.net/duplicity/+question/107216
 849 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
 850 #GPG_KEY_SIGN='<prvkey>'
 851 # set if signing key passphrase differs from encryption (key) passphrase
 852 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
 853 #GPG_PW_SIGN='<signpass>'
 854 
 855 # uncomment and set a file path or name force duply to use this gpg executable
 856 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
 857 #GPG='/usr/local/gpg-2.1/bin/gpg'
 858 
 859 # gpg options passed from duplicity to gpg process (default='')
 860 # e.g. "--trust-model pgp|classic|direct|always" 
 861 #   or "--compress-algo=bzip2 --bzip2-compress-level=9"
 862 #   or "--personal-cipher-preferences AES256,AES192,AES..."
 863 #   or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
 864 #   or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
 865 #      also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
 866 #GPG_OPTS=''
 867 
 868 # disable preliminary tests with the following setting
 869 #GPG_TEST='disabled'
 870 # disable automatic gpg key importing altogether
 871 #GPG_IMPORT='disabled'
 872 # disable automatic gpg key exporting to profile folder
 873 #GPG_EXPORT='disabled'
 874 
 875 # backend, credentials & location of the backup target (URL-Format)
 876 # generic syntax is
 877 #   scheme://[user[:password]@]host[:port]/[/]path
 878 # e.g.
 879 #   sftp://bob:secret@backupserver.com//home/bob/dupbkp
 880 # for details and available backends see duplicity manpage, section URL Format
 881 #   http://duplicity.nongnu.org/duplicity.1.html#sect7
 882 # BE AWARE:
 883 #   some backends (cloudfiles, S3 etc.) need additional env vars to be set to
 884 #   work properly, read after the TARGET definition for more details.
 885 # ATTENTION:
 886 #   characters other than A-Za-z0-9.-_.~ in the URL have to be
 887 #   replaced by their url encoded pendants, see
 888 #     http://en.wikipedia.org/wiki/Url_encoding
 889 #   if you define the credentials as TARGET_USER, TARGET_PASS below $ME
 890 #   will try to url_encode them for you if the need arises.
 891 TARGET='${DEFAULT_TARGET}'
 892 # optionally the username/password can be defined as extra variables
 893 # setting them here _and_ in TARGET results in an error
 894 # ATTENTION:
 895 #   there are backends that do not support the user/pass auth scheme.
 896 #   prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
 897 #   duplicity manpage. usually there is a NOTE section explaining if and which
 898 #   env vars should be set.
 899 #TARGET_USER='${DEFAULT_TARGET_USER}'
 900 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
 901 # e.g. for cloud files backend it might look like this (uncomment for use!)
 902 #export CLOUDFILES_USERNAME='someuser'
 903 #export CLOUDFILES_APIKEY='somekey'
 904 #export CLOUDFILES_AUTHURL ='someurl'
 905 # the following is an incomplete list (<backend>: comma separated env vars list)
 906 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
 907 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
 908 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
 909 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
 910 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
 911 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
 912 #        SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
 913 
 914 # base directory to backup
 915 SOURCE='${DEFAULT_SOURCE}'
 916 
 917 # a command that runs duplicity e.g. 
 918 #  shape bandwidth use via trickle
 919 #  "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
 920 #DUPL_PRECMD=""
 921 
 922 # override the used python interpreter, defaults to
 923 #  - parsed result of duplicity's shebang or 'python2'
 924 #   e.g. "python2" or "/usr/bin/python2.7"
 925 #PYTHON="python"
 926 
 927 # exclude folders containing exclusion file (since duplicity 0.5.14)
 928 # Uncomment the following two lines to enable this setting.
 929 #FILENAME='.duplicity-ignore'
 930 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
 931 
 932 # Time frame for old backups to keep, Used for the "purge" command.  
 933 # see duplicity man page, chapter TIME_FORMATS)
 934 #MAX_AGE=1M
 935 
 936 # Number of full backups to keep. Used for the "purgeFull" command. 
 937 # See duplicity man page, action "remove-all-but-n-full".
 938 #MAX_FULL_BACKUPS=1
 939 
 940 # Number of full backups for which incrementals will be kept for.
 941 # Used for the "purgeIncr" command.
 942 # See duplicity man page, action "remove-all-inc-of-but-n-full".
 943 #MAX_FULLS_WITH_INCRS=1
 944 
 945 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3) 
 946 # forces a full backup if last full backup reaches a specified age, for the 
 947 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
 948 # Uncomment the following two lines to enable this setting.
 949 #MAX_FULLBKP_AGE=1M
 950 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE " 
 951 
 952 # sets duplicity --volsize option (available since v0.4.3.RC7)
 953 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
 954 # VOLSIZE must be number of MB's to set the volume size to.
 955 # Uncomment the following two lines to enable this setting. 
 956 #VOLSIZE=50
 957 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
 958 
 959 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
 960 # default is 4, if not set
 961 #VERBOSITY=5
 962 
 963 # temporary file space. at least the size of the biggest file in backup
 964 # for a successful restoration process. (default is '/tmp', if not set)
 965 #TEMP_DIR=/tmp
 966 
 967 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
 968 # unencrypted meta data of the backup, enabling new incrementals without the 
 969 # need to decrypt backend metadata first. If empty or deleted somehow, the 
 970 # private key and it's password are needed.
 971 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
 972 #       big over time so you might want to put it not in the home dir.
 973 # default '~/.cache/duplicity/duply_<profile>/'
 974 # if set  '\${ARCH_DIR}/<profile>'
 975 #ARCH_DIR=/some/space/safe/.duply-cache
 976 
 977 # DEPRECATED setting
 978 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users 
 979 # to change the time separator from ':' to another character that will work 
 980 # on their system.  HINT: For Windows SMB shares, use --time-separator='_'.
 981 # NOTE: '-' is not valid as it conflicts with date separator.
 982 # ATTENTION: only use this with duplicity < 0.5.10, since then default file 
 983 #            naming is compatible and this option is pending depreciation 
 984 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
 985 
 986 # DEPRECATED setting
 987 # activates duplicity --short-filenames option, when uploading to a file
 988 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
 989 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
 990 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file 
 991 #            naming is compatible and this option is pending depreciation
 992 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
 993 
 994 # more duplicity command line options can be added in the following way
 995 # don't forget to leave a separating space char at the end
 996 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here " 
 997 
 998 EOF
 999 
1000 # create initial exclude file
1001     cat <<EOF >"$EXCLUDE"
1002 # although called exclude, this file is actually a globbing file list
1003 # duplicity accepts some globbing patterns, even including ones here
1004 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1005 # - dir/bar/foo
1006 # + dir/bar
1007 # - **
1008 # for more details see duplicity manpage, section File Selection
1009 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1010 
1011 EOF
1012 
1013   # Hints on first usage
1014   cat <<EOF
1015 
1016 Congratulations. You just created the profile '$FTPLCFG'.
1017 The initial config file has been created as 
1018 '$CONF'.
1019 You should now adjust this config file to your needs.
1020 
1021 $(hint_profile)
1022 
1023 EOF
1024 fi
1025 
1026 }
1027 
1028 # used in usage AND create_config
1029 function hint_profile {
1030   cat <<EOF
1031 IMPORTANT:
1032   Copy the _whole_ profile folder after the first backup to a safe place.
1033   It contains everything (duply related) needed to restore your backups. 
1034 
1035   Pay attention to (possibly later added) external files such as credentials
1036   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1037   (e.g. DPBX_ACCESS_TOKEN). 
1038   It is good policy to place those in the profile folder if possible at all.
1039     e.g. in case of 'multi://' target the config .json file
1040   Env vars should be added to duply profiles' conf file.
1041 
1042   Keep access to these files restricted as they contain information (gpg key,
1043   passphrases etc.) to access and modify your backups.
1044   
1045   Finally:
1046   You should attempt a restore from an unrelated host to be sure you really
1047   have everything needed for restoration.
1048 
1049   Repeat these steps after _all_ configuration changes. Some configuration 
1050   options are crucial for restoration.
1051 
1052 EOF
1053 }
1054 
1055 function separator {
1056   echo "--- $@ ---"
1057 }
1058 
1059 function inform {
1060   echo -e "\nINFO:\n\n$@\n"
1061 }
1062 
1063 function warning {
1064   echo -e "\nWARNING:\n\n$@\n"
1065 }
1066 
1067 function warning_oldhome {
1068   local old=$1 new=$2
1069   warning " ftplicity changed name to duply since you created your profiles.
1070   Please rename the old folder
1071   '$old'
1072   to
1073   '$new'
1074   and this warning will disappear.
1075   If you decide not to do so profiles will _only_ work from the old location."
1076 }
1077 
1078 function error_print {
1079   echo -e "$@" >&2
1080 }
1081 
1082 function error {
1083   error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1084   exit -1
1085 }
1086 
1087 function error_gpg {
1088   [ -n "$2" ] && local hint="\n  $2\n\n  "
1089   
1090   error "$1
1091 
1092 Hint${hint:+s}:
1093   ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1094   Don't forget the used _password_ as you will need it.
1095   When done enter the 8 digit id & the password in the profile conf file.
1096 
1097   The key id can be found doing a 'gpg --list-keys'. In the example output 
1098   below the key id for the public key would be FFFFFFFF.
1099 
1100   pub   1024D/FFFFFFFF 2007-12-17
1101   uid                  duplicity
1102   sub   2048g/899FE27F 2007-12-17
1103 "
1104 }
1105 
1106 function error_gpg_test {
1107   [ -n "$2" ] && local hint="\n  $2\n\n  "
1108 
1109   error "$1
1110 
1111 Hint${hint:+s}:
1112   ${hint}This error means that gpg is probably misconfigured or not working 
1113   correctly. The error message above should help to solve the problem.
1114   However, if for some reason $ME_NAME should misinterpret the situation you 
1115   can define GPG_TEST='disabled' in the conf file to bypass the test.
1116   Please do not forget to report the bug in order to resolve the problem
1117   in future versions of $ME_NAME.
1118 "
1119 }
1120 
1121 function error_path {
1122   error "$@
1123 PATH='$PATH'
1124 "
1125 }
1126 
1127 function error_to_string {
1128 	[ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1129 }
1130 
1131 function duplicity_version_get {
1132   # nothing to do, just print
1133   var_isset DUPL_VERSION && return
1134   
1135   local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1136   # only run with a user specific python if configured (running by default
1137   # breaks homebrew as they place a shell wrapper for duplicity in path)
1138   PYTHON_BIN="$(python_binary)" &&\
1139     CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
1140   CMD="$CMD --version 2>&1"
1141   DUPL_VERSION_OUT=$(eval "$CMD")
1142   DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1143   #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1144   DUPL_VERSION_VALUE=0
1145   DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1146   if (match(v,/[^\.0-9]+[0-9]*$/)){
1147     rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1148   if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1149   split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1150   print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1151   print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1152   print "DUPL_VERSION_RC=\047"rc"\047"
1153   print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1154   }')
1155   eval "$DUPL_VERSION_AWK"
1156   #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1157 
1158   # doublecheck findings and report error
1159   if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1160     inform "duplicity version check failed (please report, this is a bug)
1161 the command
1162   $CMD
1163 resulted in
1164   $DUPL_VERSION_OUT
1165 "
1166   elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1167     error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1168 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1169 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1170   fi
1171 }
1172 
1173 function duplicity_version_ge {
1174   [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1175 }
1176 
1177 function duplicity_version_lt {
1178   ! duplicity_version_ge "$1"
1179 }
1180 
1181 # parse interpreter from duplicity shebang
1182 function duplicity_python_binary_parse {
1183   # cached result
1184   ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
1185 
1186   # parse it or warn
1187   local DUPL_BIN=$(which duplicity)
1188   DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1189   if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1190     warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
1191 Will assume it is '$DEFAULT_PYTHON'."
1192     DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
1193   fi
1194 }
1195 
1196 function run_script { # run pre/post scripts
1197   local ERR=0
1198   local SCRIPT="$1"
1199   if [ ! -z "$PREVIEW" ] ; then	
1200     echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1201   elif [ -r "$SCRIPT" ] ; then 
1202     echo -n "Running '$SCRIPT' "
1203     if [ -x "$SCRIPT" ]; then
1204       OUT=$("$SCRIPT" 2>&1)
1205       ERR=$?
1206     else
1207       OUT=$(. "$SCRIPT" 2>&1)
1208       ERR=$?
1209     fi
1210     [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1211     echo -en ${OUT:+"Output: $OUT\n"} ;
1212   else
1213     echo "Skipping n/a script '$SCRIPT'."
1214   fi
1215   return $ERR
1216 }
1217 
1218 function run_cmd {
1219   # run or print escaped cmd string
1220   local CMD_ERR=0
1221   if [ -n "$PREVIEW" ]; then
1222     CMD_OUT=$( echo "$@ 2>&1" )
1223     CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1224   elif [ -n "$CMD_DISABLED" ]; then
1225     CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1226   else
1227     echo -n -e "$CMD_MSG"
1228     CMD_OUT=` eval "$@" 2>&1 `
1229     CMD_ERR=$?
1230     if [ "$CMD_ERR" = "0" ]; then
1231       CMD_MSG=" (OK)"
1232     else
1233       CMD_MSG=" (FAILED)"
1234     fi
1235   fi
1236   echo -e "$CMD_MSG"
1237   # reset
1238   unset CMD_DISABLED CMD_MSG
1239   return $CMD_ERR
1240 }
1241 
1242 function qw { quotewrap "$@"; }
1243 
1244 function quotewrap {
1245   local param="$@"
1246   # quote strings having non word chars (e.g. spaces)
1247   if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1248     echo "$param" | awk '{\
1249       gsub(/[\047]/,"\047\\\047\047",$0);\
1250       gsub(/[\042]/,"\047\\\042\047",$0);\
1251       print "\047"$0"\047"}'
1252     return
1253   fi
1254   echo $param
1255 }
1256 
1257 function duplicity_params_global {
1258   # already done? return
1259   var_isset 'DUPL_PARAMS_GLOBAL' && return
1260   local DUPL_ARG_ENC
1261 
1262   # use key only if set in config, else leave it to symmetric encryption
1263   if gpg_disabled; then
1264     local DUPL_PARAM_ENC='--no-encryption'
1265   else
1266     local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1267     gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1268     # interpret password settings
1269     var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1270     var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1271   fi
1272 
1273   local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1274 
1275   # set name for dupl archive folder, since 0.6.0
1276   if duplicity_version_ge 601; then
1277     local DUPL_ARCHDIR=''
1278     if var_isset 'ARCH_DIR'; then
1279       DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1280       # reuse erronously duply_ prefixed folders from bug #117
1281       if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1282         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1283       else
1284         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1285       fi
1286     else
1287       DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1288     fi
1289   fi
1290 
1291 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1292 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1293  ${GPG_OPTS}"
1294 
1295 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1296  ${DUPL_ARG_ENC}"
1297 }
1298 
1299 # function to filter the DUPL_PARAMS var from user conf
1300 function duplicity_params_conf {
1301   # reuse cmd var from main loop
1302   ## in/exclude parameters are currently not supported on restores
1303   if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1304     # filter exclude params from fetch/restore/status
1305     eval "stripXcludes $DUPL_PARAMS"
1306     return
1307   fi
1308 
1309   # nothing done, print unchanged
1310   echo "$DUPL_PARAMS"
1311 }
1312 
1313 # strip in/exclude parameters from param string
1314 function stripXcludes {
1315   local STRIPNEXT OUT;
1316   for p in "$@"; do
1317     if [ -n "$STRIPNEXT" ]; then
1318       unset STRIPNEXT
1319       # strip the value of previous parameter
1320       continue
1321     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1322       # strips e.g. --include /foo/bar
1323       STRIPNEXT="yes"
1324       continue
1325     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1326       # strips e.g. --include=/foo/bar
1327       continue
1328     fi
1329     
1330     OUT="$OUT $(qw "$p")"
1331   done
1332   echo "$OUT"
1333 }
1334 
1335 function duplify { # the actual wrapper function
1336   local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1337 
1338   # put command (with params) first in duplicity parameters
1339   for param in "$@" ; do
1340   # split cmd from params (everything before splitchar --)
1341     if [ "$param" == "--" ] ; then
1342       PARAMSNOW=1
1343     else
1344       # wrap in quotes to protect from spaces
1345       [ ! $PARAMSNOW ] && \
1346         DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1347       || \
1348         DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1349     fi
1350   done
1351 
1352   # init global duplicity parameters same for all tasks
1353   duplicity_params_global
1354 
1355   local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
1356   # run in cmd line preview mode if requested
1357   var_isset 'PREVIEW' && RUN=echo
1358   # try to resolve duplicity path for usage with python interpreter
1359   DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
1360   # only run with a user specific python if configured (running by default
1361   # breaks homebrew as they place a shell wrapper for duplicity in path)
1362   PYTHON_BIN="$(python_binary)" &&\
1363     BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
1364 
1365 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1366  ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1367  $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1368 
1369   local ERR=$?
1370   return $ERR
1371 }
1372 
1373 function secureconf { # secure the configuration dir
1374 	#PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1375 	local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1376 	if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1377 		chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1378 		warning "The profile's folder 
1379 '$CONFDIR'
1380 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1381 	fi
1382 }
1383 
1384 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1385 function date_fix {
1386 	local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1387 	local date
1388 	#[ "$1" == "%N" ] && return #test the no nsec test below
1389 	# gnu date with -d @epoch
1390 	date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1391 		echo $date && return
1392 	# date bsd,osx with -r epoch
1393 	date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1394 		echo $date && return	
1395 	# date busybox with -d epoch -D %s
1396 	date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1397 		echo $date && return
1398 	## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1399 	# python fallback
1400 	date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1401 		echo $date && return
1402 	# awk fallback
1403 	date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1404 		echo $date && return
1405 	# perl fallback
1406 	date=$(perl  -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1407 		echo $date && return
1408 	# error
1409 	echo "ERROR"
1410 	return 1
1411 }
1412 
1413 function nsecs {
1414 	local NSECS
1415 	# test if date supports nanosecond output
1416 	if ! var_isset NSECS_DISABLED; then
1417 		NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1418 		[ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1419 	fi
1420 
1421 	# add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1422 	if [ "$NSECS_DISABLED" == "1" ]; then
1423 		date_fix %s000000000
1424 	else
1425 		date_fix %s%N
1426 	fi
1427 }
1428 
1429 function nsecs_to_sec {
1430 	echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1431 }
1432 
1433 function datefull_from_nsecs {
1434 	date_from_nsecs $1 '%F %T'
1435 }
1436 
1437 function date_from_nsecs {
1438 	local FORMAT=${2:-%T}
1439 	local TIME=$(nsecs_to_sec $1)
1440 	local SECS=${TIME%.*}
1441 	local DATE=$(date_fix "%T" ${SECS:-0})
1442 	echo $DATE.${TIME#*.}
1443 }
1444 
1445 function var_isset {
1446   if [ -z "$1" ]; then
1447     echo "ERROR: function var_isset needs a string as parameter"
1448   elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1449     return 0
1450   fi
1451   return 1
1452 }
1453 
1454 function is_condition {
1455   local CMD=$(tolower "$1")
1456   [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1457 }
1458 
1459 function is_groupMarker {
1460   local CMD=$(tolower "$1")
1461   [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1462 }
1463 
1464 function is_command {
1465   local CMD=$(tolower "$1")
1466   ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1467 }
1468 
1469 function url_encode {
1470   # utilize python, silently do nothing on error - because no python no duplicity
1471   OUT=$("$(python_binary)" -c "
1472 try: import urllib.request as urllib
1473 except ImportError: import urllib
1474 print(urllib.${2}quote('$1'));
1475 " 2>/dev/null ); ERR=$?
1476   [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1477 }
1478 
1479 function url_decode {
1480   # reuse function above with a simple string param hack
1481   url_encode "$1" "un"
1482 }
1483 
1484 function toupper {
1485   echo "$@"|awk '$0=toupper($0)'
1486 }
1487 
1488 function tolower {
1489   echo "$@"|awk '$0=tolower($0)'
1490 }
1491 
1492 function isnumber {
1493   case "$*" in
1494     ''|*[!0-9]*) return 1;;
1495     *) return 0;;
1496   esac
1497 }
1498 
1499 #function tmp_space {
1500 #  
1501 #  if ! isnumber $VOLSIZE; then
1502 #    inform "failed to determine free space (please report, this is a bug)"
1503 #    return
1504 #  fi
1505 #  
1506 # get free temp space
1507 #  TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1508 #  # check for free space or FAIL
1509 #  if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1510 #    error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1511 #than one duplicity volume (${VOLSIZE}MB).
1512 #    
1513 #  Hint: Free space or change TEMP_DIR setting."
1514 #fi
1515 #
1516 #}
1517 
1518 function gpg_disabled {
1519   echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1520 }
1521 
1522 # usage: join SEPARATOR "entry1" "entry2"
1523 function join {
1524   local SEP="$1" ENTRY OUT; shift;
1525   for ENTRY in "$@"; do
1526     ENTRY=${ENTRY//$SEP/\\$SEP}
1527     [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1528   done
1529   echo $OUT
1530 }
1531 
1532 function gpg_testing {
1533   [ "$GPG_TEST" != "disabled" ]
1534 }
1535 
1536 function gpg_signing {
1537   echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1538 }
1539 
1540 function gpg_keytype {
1541   echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1542 }
1543 
1544 # parameter key id, key_type
1545 function gpg_keyfile {
1546   local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1547   local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1548   echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1549 }
1550 
1551 # parameter key id
1552 function gpg_import {
1553   local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1554   [ "$GPG_IMPORT" = "disabled" ] && {
1555     echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1556     return
1557   }
1558 
1559   # create a list of legacy key file names and current naming scheme
1560   # we always import pub and sec if they are avail in conf folder
1561   local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1562                    $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1563 
1564   # Try autoimport from existing old gpgkey files 
1565   # and new gpgkey.XXX.asc files (since v1.4.2)
1566   # and even newer gpgkey.XXX.[pub|sec].asc
1567   for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1568     FILE=${KEYFILES[$i]}
1569     if [ -f "$FILE" ]; then
1570       FOUND=1
1571       
1572       CMD_MSG="Import keyfile '$FILE' to keyring"
1573       run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1574       if [ "$?" != "0" ]; then 
1575         warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1576         ERR=1
1577         # continue with next
1578         continue
1579       fi
1580     fi
1581   done
1582 
1583   if [ "$FOUND" -eq 0 ]; then
1584     echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1585     return 1
1586   fi
1587 
1588   # try to set trust automagically
1589   CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1590   run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1591   if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then 
1592    # success on all levels, we're done
1593    return $ERR
1594   fi
1595 
1596   # failover: user has to set trust manually
1597   echo -e "For $ME_NAME to work you have to set the trust level 
1598 with the command \"trust\" to \"ultimate\" (5) now.
1599 Exit the edit mode of gpg with \"quit\"."
1600   CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1601   run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1602 
1603   return $ERR
1604 }
1605 
1606 # see 'How to specify a user ID' on gpg manpage
1607 function gpg_fingerprint {
1608   gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1609   awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1610 }
1611 
1612 function gpg_export_if_needed {
1613   [ "$GPG_EXPORT" = 'disabled' ] && { \
1614     echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1615     return
1616   }
1617 
1618   local SUCCESS FILE KEY_TYPE
1619   local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1620   for KEY_ID in "$@"; do
1621     # check if already exported, do it if not
1622     for KEY_TYPE in PUB SEC; do
1623       FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1624       if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1625 
1626         # exporting
1627         CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1628         # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1629         # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1630         run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1631         CMD_ERR=$?
1632 
1633         if [ "$CMD_ERR" = "0" ]; then
1634           CMD_MSG="Write file '"$(basename "$FILE")"'"
1635           run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1636         fi
1637 
1638         if [ "$CMD_ERR" != "0" ]; then
1639           warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1640         else
1641           SUCCESS=1
1642         fi
1643 
1644         # cleanup
1645         rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1646       fi
1647     done
1648   done
1649   
1650   [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1651 You should backup your changed profile folder now and store it in a safe place."
1652 }
1653 
1654 # replace all non-alnum chars with underscore (for file operations)
1655 function gpg_key_legalize {
1656   echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1657 }
1658 
1659 function gpg_key_cache {
1660   local RES
1661   local MODE=$1
1662   shift
1663   local PREFIX="GPG_KEY"
1664   local SUFFIX=$(gpg_key_legalize "$@")
1665   local KEYID="$*"
1666   local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1667   if [ "$MODE" = "RESET" ]; then
1668     eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1669     return 255
1670   elif ! var_isset "$CACHE"; then
1671     if [ "$MODE" = "PUB" ]; then
1672       RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1673     elif [ "$MODE" = "SEC" ]; then
1674       RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1675     else
1676       return 255
1677     fi
1678     eval $CACHE=$RES
1679   fi
1680   eval return \$$CACHE
1681 }
1682 
1683 function gpg_pub_avail {
1684   gpg_key_cache PUB "$@"
1685 }
1686 
1687 function gpg_sec_avail {
1688   gpg_key_cache SEC "$@"
1689 }
1690 
1691 function gpg_key_format {
1692   echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1693 }
1694 
1695 # splits a comma separated line into lines, respects escaped commas
1696 function gpg_split_keyset {
1697   local LIST
1698   LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1699   echo -e "$LIST"
1700 }
1701 
1702 function gpg_prefix_keyset {
1703   local PREFIX="$1" OUT=""
1704   shift
1705   for KEY_ID in "$@"; do
1706     OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1707   done
1708   echo $OUT
1709 }
1710 
1711 # grep a variable from conf text file (currently not used)
1712 function gpg_passwd {
1713   [ -r "$CONF" ] && \
1714   awk '/^[ \t]*GPG_PW[ \t=]/{\
1715         sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1716         gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1717         print $0; exit}' "$CONF"
1718 }
1719 
1720 # return success if at least one secret key is available
1721 function gpg_key_decryptable {
1722   # no keys, no problem
1723   gpg_symmetric && return 0
1724 
1725   local KEY_ID
1726   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1727     gpg_sec_avail "$KEY_ID" && return 0
1728   done
1729   return 1
1730 }
1731 
1732 function gpg_symmetric {
1733   [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1734 }
1735 
1736 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1737 function gpg_param_passwd {
1738   var_isset GPG_USEAGENT && exit 1
1739   
1740   if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1741     echo "--passphrase-fd 0 --batch"
1742   fi
1743 }
1744 
1745 # select the earliest defined and create an "echo <value> |" string
1746 function gpg_pass_pipein {
1747   var_isset GPG_USEAGENT && exit 1
1748   
1749   for var in "$@"
1750   do
1751     if var_isset "$var"; then
1752       echo "echo $(qw $(eval echo \$$var)) |"
1753       return 0
1754     fi
1755   done
1756   
1757   return 1
1758 }
1759 
1760 # checks if gpg-agent is available, returns error code
1761 # 0 on success
1762 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1763 # 2 if GPG_AGENT_INFO is stale
1764 # 3 cannot connect to gpg-agent
1765 function gpg_agent_avail {
1766   # GPG_AGENT_INFO is deprecated in gpg2.1, 
1767   # first try to connect to a possibly running agent here
1768   local ERR=3
1769   gpg-agent > /dev/null 2>&1 && return 0
1770 
1771   # detect stale pid in legacy GPG_AGENT_INFO env var
1772   if var_isset GPG_AGENT_INFO; then
1773     # check if a pid matching process is running at all
1774     local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1775     if isnumber "$GPG_AGENT_PID"; then
1776       ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1777     fi
1778   fi
1779 
1780   return $ERR
1781 }
1782 
1783 function gpg_custom_binary {
1784   var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1785     echo "--gpg-binary $(qw "$GPG")"
1786 }
1787 
1788 function gpg_binary {
1789   local BIN
1790   var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1791   echo "$BIN"
1792 }
1793 
1794 function gpg_avail {
1795   lookup "$(gpg_binary)"
1796 }
1797 
1798 # enforce the use of our selected gpg binary
1799 function gpg {
1800   command "$(gpg_binary)" "$@"
1801 }
1802 export -f gpg
1803 
1804 # start of script #######################################################################
1805 
1806 # confidentiality first, all we create is only readable by us
1807 umask 077
1808 
1809 # check if ftplicity is there & executable
1810 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1811 
1812 if [ ${#@} -eq 1 ]; then
1813   cmd="${1}"
1814 else
1815   FTPLCFG="${1}" ; cmd="${2}"
1816 fi
1817 
1818 # deal with command before profile validation calls
1819 # show requested version
1820 # OR requested usage info
1821 # OR create a profile
1822 # OR fall through
1823 ##if [ ${#@} -le 2 ]; then
1824 case "$cmd" in
1825   changelog)
1826     changelog
1827     exit 0
1828     ;;
1829   create)
1830     set_config
1831     if [ -d "$CONFDIR" ]; then
1832       error "The profile '$FTPLCFG' already exists in
1833 '$CONFDIR'.
1834 
1835 Hint:
1836  If you _really_ want to create a new profile by this name you will 
1837  have to manually delete the existing profile folder first."
1838       exit 1
1839     else
1840       create_config
1841       exit 0
1842     fi
1843     ;;
1844   txt2man)
1845     set_config
1846     usage_txt2man
1847     exit 0
1848     ;;
1849   usage|-help|--help|-h|-H)
1850     set_config
1851     usage_info
1852     exit 0
1853     ;;
1854   version|-version|--version|-v|-V)
1855     # profile can override GPG/PYTHON, so import it if it was given
1856     var_isset FTPLCFG && {
1857       set_config
1858       [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1859     }
1860     version_info_using
1861     exit 0
1862     ;;
1863   # fallthrough.. we got a command that needs an existing profile
1864   *)
1865     # if we reach here, user either forgot profile or chose wrong profileless command
1866     if [ ${#@} -le 1 ]; then
1867       error "\
1868  Missing or wrong parameters. 
1869  Only the commands 
1870    changelog, create, usage, txt2man, version
1871  can be called without selecting an existing profile first.
1872  Your command was '$cmd'.
1873 
1874  Hint: Run '$ME usage' to get help."
1875     fi
1876 esac
1877 
1878 
1879 # Hello world
1880 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1881 
1882 # check system environment
1883 
1884 # is duplicity avail
1885 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1886 # init, exec duplicity version check info
1887 duplicity_python_binary_parse
1888 duplicity_version_get
1889 
1890 # check for certain important helper programs
1891 for f in awk grep "$(python_binary)"; do
1892   lookup "$f" || \
1893     error_path "$f missing. installed und available in path?"
1894 done
1895 
1896 ### read configuration
1897 set_config
1898 # check validity
1899 if [ ! -d "$CONFDIR" ]; then 
1900     error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1901 '$CONFDIR'.
1902 
1903 Hints:
1904 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1905  Use '$ME <name> create' to create a new profile.
1906  Use '$ME usage' to get usage help."
1907 elif [ ! -x "$CONFDIR" ]; then
1908     error "\
1909 Profile folder in '$CONFDIR' cannot be accessed.
1910 
1911 Hint: 
1912  Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1913 elif [ ! -f "$CONF" ] ; then
1914   error "'$CONF' not found."
1915 elif [ ! -r "$CONF" ] ; then
1916   error "'$CONF' not readable."
1917 else
1918   . "$CONF"
1919   #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1920   TEMP_DIR=${TEMP_DIR:-'/tmp'}
1921   # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1922   if var_isset 'TARGET_PW'; then
1923     TARGET_PASS="${TARGET_PW}"
1924   fi
1925 fi
1926 echo "Using profile '$CONFDIR'."
1927 
1928 # secure config dir, if needed w/ warning
1929 secureconf
1930 
1931 # split TARGET in handy variables
1932 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1933   target=$0; match(target,/^([^\/:]+):\/\//); \
1934   prot=substr(target,RSTART,RLENGTH);\
1935   rest=substr(target,RSTART+RLENGTH); \
1936   if (credsavail=match(rest,/^[^@]*@/)){\
1937     creds=substr(rest,RSTART,RLENGTH-1);\
1938     credcount=split(creds,cred,":");\
1939     rest=substr(rest,RLENGTH+1);\
1940     # split creds with regexp\
1941     match(creds,/^([^:]+)/);\
1942     user=substr(creds,RSTART,RLENGTH);\
1943     pass=substr(creds,RSTART+1+RLENGTH);\
1944   };\
1945   # filter quotes or escape them\
1946   gsub(/[\047\042]/,"",prot);\
1947   gsub(/[\047\042]/,"",rest);\
1948   gsub(/[\047]/,"\047\\\047\047",creds);\
1949   print "TARGET_URL_PROT=\047"prot"\047\n"\
1950          "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1951          "TARGET_URL_CREDS=\047"creds"\047\n";\
1952    if(user){\
1953      gsub(/[\047]/,"\047\\\047\047",user);\
1954      print "TARGET_URL_USER=\047"user"\047\n"}\
1955    if(pass){\
1956      gsub(/[\047]/,"\047\\\047\047",pass);\
1957      print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1958   }')
1959 eval "${TARGET_SPLIT_URL}"
1960 
1961 # fetch commmand from parameters ########################################################
1962 # Hint: cmds is also used to check if authentification info sufficient in the next step 
1963 cmds="$2"; shift 2
1964 
1965 # complain if command(s) missing
1966 [ -z $cmds ] && error "  No command given.
1967 
1968   Hint: 
1969     Use '$ME usage' to get usage help."
1970 
1971 # process params
1972 for param in "$@"; do
1973   #echo !$param!
1974   case "$param" in
1975     # enable ftplicity preview mode
1976     '--preview')
1977       PREVIEW=1
1978       ;;
1979     # interpret duplicity disable encr switch
1980     '--disable-encryption')
1981       GPG_KEY='disabled'
1982       ;;
1983     *)
1984       if [ `echo "$param" | grep -e "^-"` ] || \
1985          [ `echo "$last_param" | grep -e "^-"` ] ; then
1986         # forward parameter[/option pairs] to duplicity
1987         dupl_opts["${#dupl_opts[@]}"]=${param}
1988       else
1989         # anything else must be a parameter (e.g. for fetch, ...)
1990         ftpl_pars["${#ftpl_pars[@]}"]=${param}
1991       fi
1992       last_param=${param}
1993       ;;
1994   esac
1995 done
1996 
1997 # plausibility check config - VARS & KEY ################################################
1998 # check if src, trg, trg pw
1999 # auth info sufficient 
2000 # gpg key, gpg pwd (might be empty) set in config
2001 # OR key in local gpg db
2002 # OR key can be imported from keyfile 
2003 # OR fail
2004 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
2005  error " Source Path (setting SOURCE) not set or still default value in conf file 
2006  '$CONF'."
2007 
2008 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2009  error " Backup Target (setting TARGET) not set or still default value in conf file 
2010  '$CONF'."
2011 
2012 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2013      [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2014  error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}') 
2015  are configured with different values. There can be only one.
2016  
2017  Hint: Remove conflicting setting."
2018 
2019 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2020      [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2021  error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}') 
2022  are configured with different values. There can be only one.
2023  
2024  Hint: Remove conflicting setting."
2025 fi
2026 
2027 # GPG config plausibility check1 (disabled check) #############################
2028 if gpg_disabled; then
2029   : # encryption disabled, all is well
2030 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2031   warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file 
2032 '$CONF'.
2033 Will disable encryption for duplicity now.
2034 
2035 Hint: 
2036  If you really want to use _no_ encryption you can disable this warning by 
2037  setting GPG_KEY='disabled' in conf file."
2038   GPG_KEY='disabled'
2039 fi
2040 
2041 # GPG availability check (now we know if gpg is really needed)#################
2042 if ! gpg_disabled; then 
2043   gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2044 fi
2045 
2046 # Output versions info ########################################################
2047 using_info
2048 
2049 # GPG create key settings, config check2 (needs gpg) ##########################
2050 if gpg_disabled; then
2051   : # the following tests are not necessary
2052 else
2053 
2054 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2055 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2056 
2057 # enc key still default?
2058 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then 
2059   error_gpg "Encryption Key GPG_KEY still default in conf file 
2060 '$CONF'."
2061 fi
2062 
2063 # create array of gpg encr keys, for further processing
2064 OIFS="$IFS" IFS=$'\n'
2065 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2066 IFS="$OIFS"
2067 
2068 # pw set? 
2069 # symmetric needs one, always
2070 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2071   ; then
2072   error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) 
2073 is empty/not set or still default value in conf file 
2074 '$CONF'."
2075 fi
2076 # this is a technicality, we can only pump one pass via pipe into gpg
2077 # but symmetric already always needs one for encryption
2078 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2079   [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2080   error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2081 '$CONF'.
2082 This is unfortunately impossible. For details see duplicity manpage, 
2083 section 'A Note On Symmetric Encryption And Signing'.
2084 
2085 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2086 Tip2: Use gpg-agent."
2087 fi
2088 
2089 # test - GPG KEY AVAILABILITY ##################################################
2090 
2091 # check gpg public keys availability, try import if needed
2092 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2093   KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2094   # test availability, try to import, retest
2095   if ! gpg_pub_avail "${KEY_ID}"; then
2096     echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2097     gpg_import "${KEY_ID}" PUB
2098     gpg_key_cache RESET "${KEY_ID}"
2099     gpg_pub_avail "${KEY_ID}" || { \
2100       gpg_testing && error_gpg \
2101       "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2102       "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2103   as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2104   If not you can put it there and $ME_NAME will autoimport it on the next run.
2105   Alternatively import it manually as the user you plan to run $ME_NAME with."
2106     }
2107   else
2108     echo "Public key '${KEY_ID}' found in keychain."
2109   fi
2110 done
2111 
2112 # check gpg encr secret encryption keys availability and fail
2113 # if none is available after a round of importing trials
2114 gpg_key_decryptable || \
2115 {
2116   echo "Missing secret keys for decryption in keychain."
2117   for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2118     KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2119     # test availability, try to import, retest
2120     if ! gpg_sec_avail "${KEY_ID}"; then
2121     echo "Try to import secret key '${KEY_ID}' from profile."
2122       gpg_import "${KEY_ID}" SEC
2123       gpg_key_cache RESET "${KEY_ID}"
2124     fi
2125   done
2126   gpg_key_decryptable || \
2127   {
2128     gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2129 has a secret key in the keychain. Decryption will be impossible!"
2130   }
2131 }
2132 
2133 # gpg secret sign key availability
2134 # if none set, autoset first encryption key as sign key
2135 if ! gpg_signing; then
2136   echo "Signing disabled per configuration."
2137 # try first key, if one set
2138 elif ! var_isset 'GPG_KEY_SIGN'; then
2139   KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2140   if [ -z "${KEY_ID}" ]; then
2141     echo "Signing disabled. No GPG_KEY entries in config."
2142     GPG_KEY_SIGN='disabled'
2143   else  
2144     # use avail OR try import OR fail
2145     if gpg_sec_avail "${KEY_ID}"; then
2146       GPG_KEY_SIGN="${KEY_ID}"
2147     else
2148       echo "Signing secret key '${KEY_ID}' not found."
2149       gpg_import "${KEY_ID}" SEC
2150       gpg_key_cache RESET "${KEY_ID}"
2151       if gpg_sec_avail "${KEY_ID}"; then
2152         GPG_KEY_SIGN="${KEY_ID}"
2153       fi
2154     fi
2155 
2156     # interpret sign key setting
2157     if var_isset 'GPG_KEY_SIGN'; then
2158       echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2159     else
2160       echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2161       GPG_KEY_SIGN='disabled'
2162     fi
2163   fi
2164 else
2165   KEY_ID="${GPG_KEY_SIGN}"
2166   if ! gpg_sec_avail "${KEY_ID}"; then
2167     inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2168     gpg_import "${KEY_ID}" SEC
2169     gpg_key_cache RESET "${KEY_ID}"
2170     gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2171   fi
2172 fi
2173 
2174 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2175 # key enc can deal without, but might profit from gpg-agent
2176 # if GPG_PW is not set alltogether
2177 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2178 if ! gpg_symmetric && \
2179    ( ! var_isset GPG_PW || \
2180      ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2181 
2182   if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2183     warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2184   elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2185     warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2186   elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2187     warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2188   else
2189     echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2190     GPG_USEAGENT="--use-agent"
2191   fi
2192 fi
2193 
2194 # end GPG config plausibility check2 
2195 fi
2196 
2197 # config plausibility check - SPACE ###########################################
2198 
2199 # is tmp is a folder and writable
2200 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2201 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2202 if [ "$?" != "0" ]; then
2203     error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2204 fi
2205 
2206 
2207 # get volsize, default duplicity volume size is 25MB since v0.5.07
2208 VOLSIZE=${VOLSIZE:-25}
2209 # double if asynch is on
2210 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2211 
2212 # TODO: check for enough (async= upload space and WARN only
2213 #       use function tmp_space 
2214 #echo TODO: reimplent tmp space check
2215 
2216 
2217 # test - GPG SANITY #####################################################################
2218 # if encryption is disabled, skip this whole section
2219 if gpg_disabled; then
2220   echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2221 elif ! gpg_testing; then 
2222   echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2223 else
2224 
2225 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2226 function cleanup_gpgtest { 
2227   echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2228   rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2229 }
2230 
2231 # signing enabled?
2232 if gpg_signing; then
2233   CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2234   CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2235 fi
2236 
2237 # using keys
2238 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2239 
2240   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2241     CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2242   done
2243   # check encrypting
2244   CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2245   run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2246   CMD_ERR=$?
2247 
2248   if [ "$CMD_ERR" != "0" ]; then 
2249     KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2250     [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2251   'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2252   can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2253     error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2254   fi
2255 
2256   # check decrypting
2257   CMD_MSG="Test - Decrypt"
2258   gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2259   run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2260   CMD_ERR=$?
2261 
2262   if [ "$CMD_ERR" != "0" ]; then 
2263     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2264   fi
2265 
2266 # symmetric only
2267 else
2268   # check encrypting
2269   CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2270   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2271   CMD_ERR=$?
2272   if [ "$CMD_ERR" != "0" ]; then 
2273     error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2274   fi
2275 
2276   # check decrypting
2277   CMD_MSG="Test - Decryption with passphrase"
2278   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2279   CMD_ERR=$?
2280   if [ "$CMD_ERR" != "0" ]; then 
2281     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2282   fi
2283 fi
2284 
2285 # compare original w/ decryptginal
2286 CMD_MSG="Test - Compare"
2287 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2288 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2289 CMD_ERR=$?
2290 if [ "$CMD_ERR" = "0" ]; then 
2291   cleanup_gpgtest
2292 else
2293   error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2294 fi
2295 
2296 fi # end disabled
2297 
2298 ## an empty line
2299 #echo
2300 
2301 # Exclude file is needed, create it if necessary
2302 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2303 
2304 # export only used keys, if bkp not already exists ######################################
2305 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2306 
2307 
2308 # command execution #####################################################################
2309 
2310 # urldecode url vars into plain text
2311 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2312 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2313 
2314 # defined TARGET_USER&PASS vars replace their URL pendants 
2315 # (double defs already dealt with)
2316 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2317 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2318 
2319 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2320 
2321 # issue some warnings
2322 case "$TARGET_URL_PROT_lowercase" in
2323   'cf+http')
2324     # info on missing AUTH_URL
2325     if ! var_isset 'CLOUDFILES_AUTHURL'; then
2326       inform "No CLOUDFILES_AUTHURL exported (in conf).
2327 Will use default which is probably rackspace."
2328     fi
2329     ;;
2330    'swift')
2331     # info on possibly missing AUTH_URL
2332        var_isset 'SWIFT_AUTHURL' ||\
2333        warning "\
2334 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2335     ;;
2336   'rsync')
2337     # everything in url (this backend does not support pass in env var)
2338     # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2339     # print warning older version is detected
2340     duplicity_version_lt 610 &&
2341       warning "\
2342 Duplicity version '$DUPL_VERSION' does not support providing the password as 
2343 env var for rsync backend. For security reasons you should consider to 
2344 update to a version greater than '0.6.10' of duplicity."
2345     ;;
2346 esac
2347 
2348 
2349 # for all protocols we put username in url and pass into env var 
2350 # for sec�rity reasons, we url_encode username to protect special chars
2351 # first sortout backends with special ways to handle password
2352 case "$TARGET_URL_PROT_lowercase" in
2353   'imap'|'imaps')
2354     var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2355     ;;
2356   *)
2357     # rest uses FTP_PASS var
2358     var_isset 'TARGET_URL_PASS' && \
2359       BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2360     ;;
2361 esac
2362 # insert url encoded username into target url if needed
2363 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2364   BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2365 else
2366   BACKEND_URL="$TARGET"
2367 fi
2368 
2369 
2370 # protect eval from special chars in url (e.g. open ')' in password, 
2371 # spaces in path, quotes) happens above in duplify() via quotewrap()
2372 SOURCE="$SOURCE"
2373 BACKEND_URL="$BACKEND_URL"
2374 EXCLUDE="$EXCLUDE"
2375 # since 0.7.03 --exclude-globbing-filelist is deprecated
2376 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist" 
2377 
2378 # translate backup to batch command 
2379 cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
2380 
2381 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2382 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2383   gsub(/\+/,\"_and_\",cmds);\
2384   gsub(/\-/,\"_or_\",cmds);\
2385   gsub(/\[/,\"_groupIn_\",cmds);\
2386   gsub(/\]/,\"_groupOut_\",cmds);\
2387   print cmds}")
2388 # convert cmds to array, lowercase for safety
2389 declare -a CMDS
2390 CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
2391 
2392 unset FTPL_ERR
2393 
2394 # run cmds
2395 for cmd in ${CMDS[*]};
2396 do
2397 
2398 ## init
2399 # raise index in cmd array for pre/post param
2400 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2401 
2402 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2403 
2404 # get next cmd,cnd vars
2405 nextno=$(( $CMD_NO ))
2406 while ! var_isset 'CMD_NEXT'
2407 do
2408   nextno=$(($nextno+1))
2409   if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2410     CMD_VALUE=${CMDS[$nextno]}
2411     is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2412     is_groupMarker "$CMD_VALUE" && continue
2413     CMD_NEXT="$CMD_VALUE"
2414   else
2415     CMD_NEXT='END'
2416   fi
2417 done
2418 
2419 # get prev cnd, cnds are skipped pseudocmds
2420 prevno=$(( $CMD_NO ));
2421 while ! var_isset 'CND_PREV'
2422 do
2423   prevno=$(($prevno-1))
2424   if [ "$prevno" -ge 0 ]; then
2425     CMD_VALUE=${CMDS[$prevno]}
2426     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2427     is_command "$CMD_VALUE" && break
2428   else
2429     break
2430   fi
2431 done
2432 
2433 # get prev cmd command minus skipped commands, only executed
2434 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2435 while ! var_isset 'CMD_PREV'
2436 do
2437   prevno=$(($prevno-1))
2438   if [ "$prevno" -ge 0 ]; then
2439     CMD_VALUE=${CMDS[$prevno]}
2440     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2441     is_groupMarker "$CMD_VALUE" && continue
2442     CMD_PREV="$CMD_VALUE"
2443   else
2444     CMD_PREV='START'
2445   fi
2446 done
2447 
2448 function get_cmd_skip_count {
2449   # find closing bracket, get group skip count
2450   local nextno=$CMD_NO
2451   local GRP_OPEN=0
2452   local GRP_SKIP=0
2453   local CMD_VALUE
2454   while [ "$nextno" -lt "${#CMDS[@]}" ]
2455   do
2456     nextno=$(($nextno+1))
2457     CMD_VALUE=${CMDS[$nextno]}
2458     GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2459     if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2460         break;
2461     elif [ "$CMD_VALUE" == 'groupin' ]; then
2462       GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2463     elif [ "$CMD_VALUE" == 'groupout' ]; then
2464       GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2465       if [ "$GRP_OPEN" -lt 1 ]; then
2466         break;
2467       fi
2468     fi
2469   done
2470 
2471   echo $GRP_SKIP;
2472 }
2473 
2474 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2475 unset SKIP_NOW
2476 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2477   # skip cnd/grp cmds silently
2478   is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2479   CMD_SKIP=$(($CMD_SKIP - 1))
2480   SKIP_NOW="yes"
2481 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2482   CMD_SKIP=$(get_cmd_skip_count)
2483   # incl. this "cmd"
2484   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2485   unset CMD_SKIPPED
2486   SKIP_NOW="yes"
2487 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2488   CMD_SKIP=$(get_cmd_skip_count)
2489   # incl. this "cmd"
2490   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2491   unset CMD_SKIPPED
2492   SKIP_NOW="yes"
2493 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2494   unset 'CMD_SKIP';
2495   SKIP_NOW="yes"
2496 fi
2497 
2498 # let's do the skip now
2499 if [ -n "$SKIP_NOW" ]; then
2500   # sum up how many commands we actually skipped for the prev var routines
2501   CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2502   continue
2503 fi
2504 
2505 # save start time
2506 RUN_START=$(nsecs)
2507 
2508 # export some useful env vars for external scripts/programs to use
2509 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2510        TARGET_URL_USER TARGET_URL_PASS \
2511        GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2512        GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2513        RUN_START
2514 
2515 # user info
2516 echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
2517 
2518 case "$(tolower $cmd)" in
2519   'pre'|'post')
2520     if [ "$cmd" == 'pre' ]; then
2521       script=$PRE
2522     else
2523       script=$POST
2524     fi
2525     # script execution in a subshell, protect us from failures/var overwrites
2526     ( run_script "$script" )
2527     ;;
2528   'bkp')
2529     duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2530           "$SOURCE" "$BACKEND_URL"
2531     ;;
2532   'incr')
2533     duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2534           "$SOURCE" "$BACKEND_URL"
2535     ;;
2536   'full')
2537     duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2538           "$SOURCE" "$BACKEND_URL"
2539     ;;
2540   'verify')
2541     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2542     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2543           "$BACKEND_URL" "$SOURCE"
2544     ;;
2545   'verifypath')
2546     TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2547     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2548     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2549   
2550   Hint: 
2551     Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2552 
2553     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2554           --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2555     ;;
2556   'list')
2557     # time param exists since 0.5.10+
2558     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2559     duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2560     ;;
2561   'cleanup')
2562     duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2563     ;;
2564   'purge')
2565     MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2566     [ -z "$MAX_AGE" ] && error "  Missing parameter <max_age>. Can be set in profile or as command line parameter."
2567     
2568     duplify remove-older-than "${MAX_AGE}" \
2569           -- "${dupl_opts[@]}" "$BACKEND_URL"
2570     ;;
2571   'purgefull')
2572     MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2573     [ -z "$MAX_FULL_BACKUPS" ] && error "  Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2574   
2575     duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2576           -- "${dupl_opts[@]}" "$BACKEND_URL"
2577     ;;
2578   'purgeincr')
2579     MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2580     [ -z "$MAX_FULLS_WITH_INCRS" ] && error "  Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2581   
2582     duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2583           -- "${dupl_opts[@]}" "$BACKEND_URL"
2584     ;;
2585   'restore')
2586     OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2587     [ -z "$OUT_PATH" ] && error "  Missing parameter target_path for restore.
2588   
2589   Hint: 
2590     Syntax is -> $ME <profile> restore <target_path> [<age>]"
2591     
2592     duplify  -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2593     ;;
2594   'fetch')
2595     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2596     TIME="${ftpl_pars[2]:-now}";
2597     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <src_path> or <target_path> for fetch.
2598   
2599   Hint: 
2600     Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2601     
2602     # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2603     duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2604               --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2605     ;;
2606   'status')
2607     duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2608     ;;    
2609   *)
2610     error "  Unknown command '$cmd'.
2611 
2612   Hint:
2613     Use '$ME usage' to get usage help."
2614     ;;
2615 esac
2616 
2617 CMD_ERR=$?
2618 RUN_END=$(nsecs)
2619 RUNTIME=$(( $RUN_END - $RUN_START ))
2620 
2621 # print message on error; set error code
2622 if [ "$CMD_ERR" -ne 0 ]; then
2623 	error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2624 	FTPL_ERR=1
2625 fi
2626 
2627 separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
2628 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2629 
2630 done
2631 
2632 exit ${FTPL_ERR}