Sourcecode / SVN

From duply (simple duplicity) - a duplicity shell frontend
(Redirected from Duply-code)
Jump to navigation Jump to search


duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/

SVN access

svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply

SVN Log

https://sourceforge.net/p/ftplicity/code/commit_browser

Latest Development Snapshot

mod time 2020-12-30

plain/text -> duply.sh

   1 #!/usr/bin/env bash
   2 #
   3 ################################################################################
   4 #  duply (grown out of ftplicity), is a shell front end to duplicity that      #
   5 #  simplifies the usage by managing settings for backup jobs in profiles.      #
   6 #  It supports executing multiple commands in a batch mode to enable single    #
   7 #  line cron entries and executes pre/post backup scripts.                     #
   8 #  Since version 1.5.0 all duplicity backends are supported. Hence the name    #
   9 #  changed from ftplicity to duply.                                            #
  10 #  See http://duply.net or http://ftplicity.sourceforge.net/ for more info.    #
  11 #  (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany            #
  12 #  (c) 2008-2020 Edgar Soldin (changes since version 1.3)                      #
  13 ################################################################################
  14 #  LICENSE:                                                                    #
  15 #  This program is licensed under GPLv2.                                       #
  16 #  Please read the accompanying license information in gpl-2.0.txt.            #
  17 ################################################################################
  18 #  TODO/IDEAS/KNOWN PROBLEMS:
  19 #  - possibility to restore time frames (incl. deleted files)
  20 #    realizable by listing each backup and restore from 
  21 #    oldest to the newest, problem: not performant
  22 #  - search file in all backups function and show available
  23 #    versions with backups date (list old avail since 0.6.06)
  24 #  - edit profile opens conf file in vi 
  25 #  - implement log-fd interpretation
  26 #  - add a duplicity option check against the options pending 
  27 #    deprecation since 0.5.10 namely --time-separator
  28 #                               --short-filenames
  29 #                              --old-filenames
  30 #  - add 'exclude_<command>' list usage e.g. exclude_verify
  31 #  - featreq 25: a download/install duplicity option
  32 #  - hint on install software if a piece is missing
  33 #  - import/export profile from/to .tgz function !!!
  34 #
  35 #  CHANGELOG:
  36 #  2.3 (30.12.2020)
  37 #  - don't import whole key pair anymore if only pub/sec is requested
  38 #  - gpg import routine informs on missing key files in profile now
  39 #  - add check/import needed secret key for decryption
  40 #  - featreq 50: Disable GPG key backups, implemented/added settings
  41 #      GPG_IMPORT/GPG_EXPORT='disabled' to conf template
  42 #
  43 #  2.2.2 (24.02.2020)
  44 #  - bugfix 120: Failures in "Autoset trust of key" during restore 
  45 #    because of gpg2.2 fingerprint output change
  46 #
  47 #  2.2.1 (22.01.2020)
  48 #  - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  49 #  - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  50 #  - fix abbreviation spelling of 'e.g.'
  51 #
  52 #  2.2 (30.12.2018)
  53 #  - featreq 44: implement grouping for batch commands
  54 #      new separators are [] (square brackets) or groupIn/groupOut
  55 #      command 'backup' translates now to [pre_bkp_post] to be skipped as
  56 #      one block in case a condition was set in the batch instruction
  57 #
  58 #  2.1 (23.07.2018)
  59 #  - be more verbose when duplicity version detection fails
  60 #  - using info shows python binary's path for easier identification now
  61 #  - reworked python interpreter handling, it's either
  62 #      configured per PYTHON var
  63 #      unconfigured, parsed from duplicity shebang
  64 #      or set to current duplicity default 'python2' (was 'python' until now)
  65 #  - do not quotewrap strings because of slashes (e.g. paths) anymore
  66 #  - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  67 #
  68 #  2.0.4 (20.02.2018)
  69 #  - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  70 #  - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  71 #  - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  72 #  - bugfix debian 882159: duply: occasionally shows negative runtimes
  73 #
  74 #  2.0.3 (29.08.2017)
  75 #  - bugfix: "line 2231: CMDS: bad array subscript"
  76 #  - bugfix 112: "env: illegal option -- u" on MacOSX
  77 #
  78 #  2.0.2 (23.05.2017)
  79 #  - bugfix: never insert creds into file:// targets
  80 #  - bugfix: avail profiles hint sometimes shortend the names by one char
  81 #  - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  82 #  - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  83 #  - bugfix 97: Unknown command should be ERROR, not WARNING 
  84 #
  85 #  2.0.1 (16.11.2016)
  86 #  - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
  87 #
  88 #  2.0 (27.10.2016)
  89 #  made this a major version change, as we broke backward compatibility anyway
  90 #  (see last change in v1.10). got complaints that rightfully pointed out
  91 #  that should only come w/ a major version change. so, here we go ;)
  92 #  if your backend stops working w/ this version create a new profile and
  93 #  export the env vars needed as described in the comments of the conf file
  94 #  directly above the SOURCE setting.
  95 #  Changes:
  96 #  - making sure multi spaces in TARGET survive awk processing
  97 #  - new env var PROFILE exported to scripts 
  98 #  - fix 102: expose a unique timestamp variable for pre/post scripts
  99 #    actually a featreq. exporting RUN_START nanosec unix timestamp
 100 #  - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
 101 #  - fix 96: duply cannot handle two consecutive spaces in paths
 102 #
 103 #  1.11.3 (29.5.2016)
 104 #  - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
 105 #
 106 #  1.11.2 (11.2.2016)
 107 #  - fix "gpg: unsafe" version print out 
 108 #  - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
 109 #  - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional 
 110 #    documentation about needed env vars to template conf file
 111 #
 112 #  1.11.1 (18.12.2015)
 113 #  - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
 114 #  - reverted duply's default PYTHON to 'python'
 115 #
 116 #  1.11 (24.11.2015)
 117 #  - remove obsolete --ssh-askpass routine
 118 #  - add PYTHON conf var to allow global override of used python interpreter
 119 #  - enforced usage of "python2" in PATH as default interpreter for internal
 120 #    use _and_ to run duplicity (setup.py changed the shebang to the fixed
 121 #    path /usr/bin/python until 0.7.05, which we circumvent this way)
 122 #  - featreq 36: support gpg-connect-agent as a means to detect if an agent is 
 123 #    running (thx Thomas Harning Jr.), used gpg-agent for detection though
 124 #  - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
 125 #  - key export routine respects gpg-agent usage now
 126 #
 127 #  1.10.1 (19.8.2015)
 128 #  - bugfix 86: Duply+Swift outputs warning
 129 #  - bugfix 87: Swift fails without BACKEND_URL
 130 #
 131 #  1.10 (31.7.2015)
 132 #  - featreq 37: busybox issues - fix awk, grep version detection,
 133 #    fix grep failure because --color=never switch is unsupported
 134 #    (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
 135 #  - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
 136 #    (thx Joachim Wiedorn, also for maintaining the debian package)
 137 #  - implemented base-/dirname as bash functions
 138 #  - featreq 31 " Support for duplicity Azure backend " - ignored a 
 139 #    contributed patch by Scott McKenzie and instead opted for removing almost
 140 #    all code that deals with special env vars required by backends.
 141 #    adding and modifying these results in too much overhead so i dropped this
 142 #    feature. the future alternative for users is to consult the duplicity 
 143 #    manpage and add the needed export definitions to the conf file.
 144 #    appended a commented example to the template conf below the auth section.
 145 #
 146 #  1.9.2 (21.6.2015)
 147 #  - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
 148 #  - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
 149 #  - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
 150 #  - added GPG conf var (see conf template for details)
 151 #  - added grep version output as it is an integral needed binary
 152 #  - added PYTHONPATH printout in version output
 153 #
 154 #  1.9.1 (13.10.2014)
 155 #  - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
 156 #  - bugfix: CMD_PREV contained command even if it was skipped
 157 #
 158 #  1.9.0 (24.8.2014)
 159 #  - bugfix: env vars were not exported when external script was executable
 160 #  - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) 
 161 #    see gpg manpage, section "How to specify a user ID"
 162 #    let gpg complain when the delivered values are invalid for whatever reason
 163 #  - started to rework tmp space checking, exposed folder & writable check
 164 #    TODO: reimplement enough file space available checking
 165 #
 166 #  1.8.0 (13.7.2014)
 167 #  - add command verifyPath to expose 'verify --file-to-restore' action
 168 #  - add time parameter support to verify command
 169 #  - add section time formats to usage output 
 170 #
 171 #  1.7.4 (24.6.2014)
 172 #  - remove ubuntu one support, service is discontinued
 173 #  - featreq 31: add authenticated swift (contributed by Justus Seifert)
 174 #
 175 #  1.7.3 (3.4.2014)
 176 #  - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
 177 #
 178 #  1.7.2 (1.4.2014 "April,April")
 179 #  - bugfix: debian Bug#743190 "duply no longer allows restoration without 
 180 #     gpg passphrase in conf file"
 181 #     GPG_AGENT_INFO env var is now needed to trigger --use-agent
 182 #  - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
 183 #
 184 #  1.7.1 (30.3.2014)
 185 #  - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to 
 186 #     incompatibility with new minus batch separator 
 187 #
 188 #  1.7.0 (20.3.2014)
 189 #  - disabled gpg key id plausibility check, too many valid possibilities
 190 #  - featreq 7 "Halt if precondition fails":
 191 #     added and(+), or(-) batch command(separator) support
 192 #  - featreq 26 "pre/post script with shebang line": 
 193 #     if a script is flagged executable it's executed in a subshell 
 194 #     now as opposed to sourced to bash, which is the default
 195 #  - bugfix: do not check if dpbx, swift credentials are set anymore 
 196 #  - bugfix: properly escape profile name, archdir if used as arguments
 197 #  - add DUPL_PRECMD conf setting for use with e.g. trickle
 198 #
 199 #  1.6.0 (1.1.2014)
 200 #  - support gs backend
 201 #  - support dropbox backend
 202 #  - add gpg-agent support to gpg test routines
 203 #  - autoenable --use-agent if passwords were not defined in config
 204 #  - GPG_OPTS are now honored everywhere, keyrings or complete gpg
 205 #    homedir can thus be configured to be located anywhere
 206 #  - always import both secret and public key if avail from config profile
 207 #  - new explanatory comments in initial exclude file
 208 #  - bugfix 7: Duply only imports one key at a time 
 209 #
 210 #  1.5.11 (19.07.2013)
 211 #  - purge-incr command for remove-all-inc-of-but-n-full feature added
 212 #    patch provided by Moritz Augsburger, thanks!
 213 #  - documented version command in man page
 214 #
 215 #  1.5.10 (26.03.2013)
 216 #  - minor indent and documentation fixes
 217 #  - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
 218 #  - bugfix: fix url_decoding generally and for python3
 219 #  - bugfix 3609075: wrong script results in status line (thx David Epping)
 220 #
 221 #  1.5.9 (22.11.2012)
 222 #  - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
 223 #  - restore/fetch now also ignores --include* or --exclude='foobar' 
 224 #
 225 #  1.5.8 (26.10.2012)
 226 #  - bugfix 3575487: implement proper cloud files support
 227 #
 228 #  1.5.7 (10.06.2012)
 229 #  - bugfix 3531450: Cannot use space in target URL (file:///) anymore
 230 #
 231 #  1.5.6 (24.5.2012)
 232 #  - commands purge, purge-full have no default value anymore for security 
 233 #    reasons; instead max value can be given via cmd line or must be set
 234 #    in profile; else an error is shown.
 235 #  - minor man page modifications
 236 #
 237 #  versioning scheme will be simplified to [major].[minor].[patch] version
 238 #  with the next version raise
 239 #
 240 #  1.5.5.5 (4.2.2012)
 241 #  - bugfix 3479605: SEL context confused profile folder's permission check
 242 #  - colon ':' in url passphrase got ignored, added python driven url_decoding
 243 #    for user & pass to better process special chars
 244 #
 245 #  1.5.5.4 (16.10.2011)
 246 #  - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
 247 #  - add support for separate sign passphrase (needs duplicity 0.6.14+)
 248 #
 249 #  1.5.5.3 (1.10.2011)
 250 #  - bugfix 3416690: preview threw echo1 error
 251 #  - fix unknown cmds error usage & friends if more than 2 params were given
 252 #
 253 #  1.5.5.2 (23.9.2011)
 254 #  - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
 255 #  - bugfix: mawk does not support \W and did not split multikey definitions
 256 #  - all parameters should survive  single (') and double (") quotes now
 257 #
 258 #  1.5.5.1 (7.6.2011)
 259 #  - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
 260 #  - bugfix 3312208: signing detection broke symmetric gpg test routine
 261 #
 262 #  1.5.5 (2.5.2011)
 263 #  - bugfix: fetch problem with space char in path, escape all params 
 264 #    containing non word chars
 265 #  - list available profiles, if given profile cannot be found
 266 #  - added --use-agent configuration hint
 267 #  - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke 
 268 #    fetch/restore
 269 #  - version command now prints out 'using installed' info
 270 #  - featreq 3166169: autotrust imported keys, based on code submitted by 
 271 #    Martin Ellis - imported keys are now automagically trusted ultimately 
 272 #  - new txt2man feature to create manpages for package maintainers
 273 #
 274 #  1.5.4.2 (6.1.2011)
 275 #  - new command changelog
 276 #  - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
 277 #  - bugfix: freebsd awk hangs on 'awk -W version' 
 278 #  - bugfix 3150244: mawk does not know '--version'
 279 #  - minor help text improvements
 280 #  - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
 281 #
 282 #  1.5.4.1 (4.12.2010)
 283 #  - output awk, python, bash version now in prolog
 284 #  - shebang uses /usr/bin/env now for freebsd compatibility, 
 285 #    bash not in /bin/bash 
 286 #  - new --disable-encryption parameter, 
 287 #    to override profile encr settings for one run
 288 #  - added exclude-if-present setting to conf template
 289 #  - bug 3126972: GPG_PW only needed for signing/symmetric encryption 
 290 #    (even though duplicity still needs it)
 291 #
 292 #  1.5.4 (15.11.2010)
 293 #  - as of 1.5.3 already, new ARCH_DIR config option
 294 #  - multiple key support
 295 #  - ftplicity-Feature Requests-2994929: separate encryption and signing key
 296 #  - key signing of symmetric encryption possible (duplicity patch committed)
 297 #  - gpg tests disable switch
 298 #  - gpg tests now previewable and more intelligent
 299 #
 300 #  1.5.3 (1.11.2010)
 301 #  - bugfix 3056628: improve busybox compatibility, grep did not have -m param
 302 #  - bugfix 2995408: allow empty password for PGP key
 303 #  - bugfix 2996459: Duply erroneously escapes '-' symbol in username
 304 #  - url_encode function is now pythonized
 305 #  - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
 306 #  - feature 3059262: Make pre and post aware of parameters, 
 307 #                     internal parameters + CMD of pre or post 
 308 #
 309 #  1.5.2.3 (16.4.2010)
 310 #  - bugfix: date again, should now work virtually anywhere
 311 #
 312 #  1.5.2.2 (3.4.2010)
 313 #  - minor bugfix: duplicity 0.6.8b version string now parsable
 314 #  - added INSTALL.txt
 315 #
 316 #  1.5.2.1 (23.3.2010)
 317 #  - bugfix: date formatting is awked now and should work on all platforms
 318 #
 319 #  1.5.2 (2.3.2010)
 320 #  - bugfix: errors print to STD_ERR now, failed tasks print an error message
 321 #  - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
 322 #  - simplified & cleaned profileless commands, removed second instance
 323 #  - generalized separator time routines
 324 #  - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
 325 #  - minor fixes
 326 #
 327 #  1.5.1.5 (5.2.2010)
 328 #  - bugfix: added special handling of credentials for rsync, imap(s)
 329 #
 330 #  1.5.1.4 (7.1.2010)
 331 #  - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
 332 #  - check if ncftp binary is available if url protocol is ftp
 333 #  - bugfix: duplicity output is now printed to screen directly to resolve
 334 #            'mem alloc problem' bug report
 335 #  - bugfix: passwords will not be in the url anymore to solve the 'duply shows
 336 #            sensitive data in process listing' bug report
 337 #
 338 #  1.5.1.3 (24.12.2009) 'merry xmas'
 339 #  - bugfix: gpg pass now apostrophed to allow space and friends
 340 #  - bugfix: credentials are now url encoded to allow special chars in them
 341 #            a note about url encoding has been added to the conf template
 342 #
 343 #  1.5.1.2 (1.11.2009)
 344 #  - bugfix: open parenthesis in password broke duplicity execution
 345 #  - bugfix: ssh/scp backend does not always need credentials e.g. key auth
 346 #
 347 #  1.5.1.1 (21.09.2009)
 348 #  - bugfix: fixed s3[+http] TARGET_PASS not needed routine
 349 #  - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
 350 #    see https://sourceforge.net/tracker/index.php?func=detail&
 351 #                aid=2864410&group_id=217745&atid=1041147
 352 #
 353 #  1.5.1 (21.09.2009) - duply (fka. ftplicity)
 354 #  - first things first: ftplicity (being able to support all backends since 
 355 #    some time) will be called duply (fka. ftplicity) from now on. The addendum
 356 #    is for the time being to circumvent confusion.
 357 #  - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity 
 358 #            command failed
 359 #  - s3[+http] now supported natively by translating user/pass to access_key/
 360 #    secret_key environment variables needed by duplicity s3 boto backend 
 361 #  - bugfix: additional output lines do not confuse version check anymore
 362 #  - list command supports now age parameter (patch by stefan on feature 
 363 #    request tracker)
 364 #  - bugfix: option/param pairs are now correctly passed on to duplicity
 365 #  - bugfix: s3[+http] needs no TARGET_PASS if command is read only
 366 #
 367 #  1.5.0.2 (31.07.1009)
 368 #  - bugfix: insert password in target url didn't work with debian mawk
 369 #            related to previous bug report
 370 #
 371 #  1.5.0.1 (23.07.2009)
 372 #  - bugfix: gawk gensub dependency raised an error on debian's default mawk
 373 #            replaced with match/substr command combination (bug report)
 374 #            https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
 375 #            group_id=217745
 376 #
 377 #  1.5.0 (01.07.2009)
 378 #  - removed ftp limitation, all duplicity backends should work now
 379 #  - bugfix: date for separator failed on openwrt busybox date, added a 
 380 #    detecting workaround, milliseconds are not available w/ busybox date
 381 #
 382 #  1.4.2.1 (14.05.2009)
 383 #  - bugfix: free temp space detection failed with lvm, fixed awk parse routine
 384 #
 385 #  1.4.2 (22.04.2009)
 386 #  - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
 387 #    armored ascii nature, the id helps if the key is switched for some reason
 388 #    im/export routines are updated accordingly (import is backward compatible 
 389 #    to the old profile/gpgkey files)         
 390 #  - profile argument is treated as path if it contains slashes 
 391 #    (for details see usage)
 392 #  - non-ftplicity options (all but --preview currently) are now passed 
 393 #    on to duplicity 
 394 #  - removed need for stat in secure_conf, it is ls based now
 395 #  - added profile folder readable check
 396 #  - added gpg version & home info output
 397 #  - awk utility availability is now checked, because it was mandatory already
 398 #  - tmp space is now checked on writability and space requirement
 399 #    test fails on less than 25MB or configured $VOLSIZE, 
 400 #    test warns if there is less than two times $VOLSIZE because 
 401 #    that's required for --asynchronous-upload option  
 402 #  - gpg functionality is tested now before executing duplicity 
 403 #    test drive contains encryption, decryption, comparison, cleanup
 404 #    this is meant to detect non trusted or other gpg errors early
 405 #  - added possibility of doing symmetric encryption with duplicity
 406 #    set GPG_KEY="" or simply comment it out
 407 #  - added hints in config template on the depreciation of 
 408 #    --short-filenames, --time-separator duplicity options
 409 #
 410 #  new versioning scheme 1.4.2b => 1.4.2, 
 411 #  beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
 412 #  to the first bug fixing version and 1.4.2.2 to the second and so on
 413 #  also the releases will now have a release date formatted (Day.Month.Year)
 414 #
 415 #  1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
 416 #            named exactly as the profile if existing in executing dir
 417 #          - improved plausibility checking of config and profile folder
 418 #          - secure_conf only acts if needed and prints a warning now
 419 #
 420 #  1.4.1b  - introduce status (duplicity collection-status) command
 421 #          - pre/post script output printed always now, not only on errors
 422 #          - new config parameter GPG_OPTS to pass gpg options
 423 #            added examples & comments to profile template conf
 424 #          - reworked separator times, added duration display
 425 #          - added --preview switch, to preview generated command lines
 426 #          - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
 427 #            profiles because they have reasonable defaults now if not set
 428 #
 429 #  1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
 430 #            therefore backup translates to pre_bkp_post now
 431 #          - bugfix: new command bkp, which represents duplicity's 
 432 #            default action (incr or full if full_if_older matches
 433 #            or no earlier backup chain is found)
 434 #
 435 #  new versioning scheme 1.4 => 1.4.0, added new minor revision number
 436 #  this is meant to slow down the rapid version growing but still keep 
 437 #  versions cleanly separated.
 438 #  only additional features will raise the new minor revision number. 
 439 #  all releases start as beta, each bugfix release will raise the beta 
 440 #  count, usually new features arrive before a version 'ripes' to stable
 441 #    
 442 #  1.4.0b
 443 #    1.4b  - added startup info on version, time, selected profile
 444 #          - added time output to separation lines
 445 #          - introduced: command purge-full implements duplicity's 
 446 #            remove-all-but-n-full functionality (patch by unknown),
 447 #            uses config variable $MAX_FULL_BACKUPS (default = 1)
 448 #          - purge config var $MAX_AGE defaults to 1M (month) now 
 449 #          - command full does not execute pre/post anymore
 450 #            use batch command pre_full_post if needed 
 451 #          - introduced batch mode cmd1_cmd2_etc
 452 #            (in turn removed the bvp command)
 453 #          - unknown/undefined command issues a warning/error now
 454 #          - bugfix: version check works with 0.4.2 and older now
 455 #    1.3b3 - introduced pre/post commands to execute/debug scripts
 456 #          - introduced bvp (backup, verify, purge)
 457 #          - bugfix: removed need for awk gensub, now mawk compatible
 458 #    1.3b2 - removed pre/post need executable bit set 
 459 #          - profiles now under ~/.ftplicity as folders
 460 #          - root can keep profiles in /etc/ftplicity, folder must be
 461 #            created by hand, existing profiles must be moved there
 462 #          - removed ftplicity in path requirement
 463 #          - bugfix: bash < v.3 did not know '=~'
 464 #          - bugfix: purge works again 
 465 #    1.3   - introduces multiple profiles support
 466 #          - modified some script errors/docs
 467 #          - reordered gpg key check import routine
 468 #          - added 'gpg key id not set' check
 469 #          - added error_gpg (adds how to setup gpg key howto)
 470 #          - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
 471 #          - duplicity_version_check routine introduced
 472 #          - added time separator, shortnames, volsize, full_if_older 
 473 #            duplicity options to config file (inspired by stevie 
 474 #            from http://weareroot.de) 
 475 #    1.1.1 - bugfix: encryption reactivated
 476 #    1.1   - introduced config directory
 477 #    1.0   - first release
 478 ################################################################################
 479 
 480 # utility functions overriding binaries
 481 
 482 # wrap grep to override possible env set GREP_OPTIONS=--color=always
 483 function grep {
 484   command env "GREP_OPTIONS=" grep "$@"
 485 }
 486 
 487 # implement basename in plain bash
 488 function basename {
 489   local stripped="${1%/}"
 490   echo "${stripped##*/}"
 491 }
 492 
 493 # implement dirname in plain bash
 494 function dirname {
 495   echo ${1%/*}
 496 }
 497 
 498 # implement basic which in plain bash
 499 function which {
 500   type -p "$@"
 501 }
 502 
 503 # check availability of executables via file name or file paths
 504 function lookup {
 505   local bin="$1"
 506   # look for file names in path via bash hash OR 
 507   # look for executables at given relative/absolute location
 508   ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
 509 }
 510 
 511 # the python binary to use, exit code 0 when configured, else 1
 512 function python_binary {
 513   # if unset, parse from duplicity shebang
 514   if ! var_isset 'PYTHON'; then
 515     duplicity_python_binary_parse;
 516     echo $DUPL_PYTHON_BIN;
 517     return 1;
 518   else
 519     # tell if PYTHON was configured manually
 520     echo $PYTHON;
 521     return 0
 522   fi
 523 }
 524 
 525 # important definitions #######################################################
 526 
 527 ME_LONG="$0"
 528 ME="$(basename $0)"
 529 ME_NAME="${ME%%.*}"
 530 ME_VERSION="2.3"
 531 ME_WEBSITE="http://duply.net"
 532 
 533 # default config values
 534 DEFAULT_SOURCE='/path/of/source'
 535 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
 536 DEFAULT_TARGET_USER='_backend_username_'
 537 DEFAULT_TARGET_PASS='_backend_password_'
 538 DEFAULT_GPG='gpg'
 539 DEFAULT_GPG_KEY='_KEY_ID_'
 540 DEFAULT_GPG_PW='_GPG_PASSWORD_'
 541 DEFAULT_PYTHON='python2'
 542 
 543 # function definitions ##########################
 544 
 545 function set_config { # sets global config vars
 546   local CONFHOME_COMPAT="$HOME/.ftplicity"
 547   local CONFHOME="$HOME/.duply"
 548   local CONFHOME_ETC_COMPAT="/etc/ftplicity"
 549   local CONFHOME_ETC="/etc/duply"
 550 
 551   # confdir can be delivered as path (must contain /)
 552   if [ `echo $FTPLCFG | grep /` ] ; then 
 553     CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
 554               ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
 555                && echo $(pwd)/${FTPLCFG} ) || \
 556               echo ${FTPLCFG})          
 557   # or DEFAULT in home/.duply folder (NEW)
 558   elif [ -d "${CONFHOME}" ]; then
 559     CONFDIR="${CONFHOME}/${FTPLCFG}"
 560   # or in home/.ftplicity folder (OLD)
 561   elif [ -d "${CONFHOME_COMPAT}" ]; then
 562     CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
 563     warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
 564   # root can put profiles under /etc/duply (NEW) if path exists
 565   elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
 566     CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
 567   # root can keep profiles under /etc/ftplicity (OLD) if path exists
 568   elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
 569     CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
 570     warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
 571   # hmm no profile folder there, then use default for error later
 572   else
 573     CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
 574   fi
 575 
 576   # remove trailing slash, get profile name etc.
 577   CONFDIR="${CONFDIR%/}"
 578   PROFILE="${CONFDIR##*/}"
 579   CONF="$CONFDIR/conf"
 580   PRE="$CONFDIR/pre"
 581   POST="$CONFDIR/post"
 582   EXCLUDE="$CONFDIR/exclude"
 583   KEYFILE="$CONFDIR/gpgkey.asc"
 584 }
 585 
 586 function version_info { # print version information
 587   cat <<END
 588   $ME_NAME version $ME_VERSION
 589   ($ME_WEBSITE)
 590 END
 591 }
 592 
 593 function version_info_using { 
 594   cat <<END
 595 $(version_info)
 596 
 597   $(using_info)
 598 END
 599 }
 600 
 601 function using_info {
 602   # init needed vars into global name space
 603   lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
 604   local NOTFOUND="INVALID"
 605   local AWK_VERSION GREP_VERSION PYTHON_RUNNER
 606   # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
 607   AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 608   GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 609   PYTHON_RUNNER=$(python_binary)
 610   local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
 611   local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
 612   local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
 613   # print out
 614   echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
 615 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
 616 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
 617 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
 618 }
 619 
 620 function usage_info { # print usage information
 621 
 622   cat <<USAGE_EOF
 623 VERSION:
 624 $(version_info)
 625   
 626 DESCRIPTION: 
 627   Duply deals as a wrapper for the mighty duplicity magic.
 628   It simplifies running duplicity with cron or on command line by:
 629 
 630     - keeping recurring settings in profiles per backup job
 631     - enabling batch operations e.g. backup_verify+purge
 632     - executing pre/post scripts (different actions possible 
 633       depending on previous or next command or it's exit status)
 634     - precondition checking for flawless duplicity operation
 635 
 636   For each backup job one configuration profile must be created.
 637   The profile folder will be stored under '~/.${ME_NAME}/<profile>'
 638   (where ~ is the current users home directory).
 639   Hint:  
 640    If the folder '/etc/${ME_NAME}' exists, the profiles for the super
 641    user root will be searched & created there.
 642 
 643 USAGE:
 644   first time usage (profile creation):  
 645     $ME <profile> create
 646 
 647   general usage in single or batch mode (see EXAMPLES):  
 648     $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
 649 
 650   For batches the conditional separators can also be written as pseudo commands
 651   and(+), or(-). See SEPARATORS for details.
 652 
 653   Non $ME options are passed on to duplicity (see OPTIONS).
 654   All conf parameters can also be defined in the environment instead.
 655 
 656 PROFILE:
 657   Indicated by a path or a profile name (<profile>), which is resolved 
 658   to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
 659 
 660   Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
 661   the folder manually before running $ME_NAME as superuser.
 662   Note:  
 663     Already existing profiles in root's home folder will cease to work
 664     unless they are moved to the new location manually.
 665 
 666   example 1:   $ME humbug backup
 667 
 668   Alternatively a _path_ might be used e.g. useful for quick testing, 
 669   restoring or exotic locations. Shell expansion should work as usual.
 670   Hint:  
 671     The path must contain at least one path separator '/', 
 672     e.g. './test' instead of only 'test'.
 673 
 674   example 2:   $ME ~/.${ME_NAME}/humbug backup
 675 
 676 SEPARATORS:
 677   _ (underscore)  
 678              neutral separator
 679   + (plus sign), _and_  
 680              conditional AND
 681              the next command will only be executed if the previous succeeded
 682   - (minus sign), _or_  
 683              conditional OR
 684              the next command will only be executed if the previous failed
 685   [] (square brackets), _groupIn_/_groupOut_  
 686              enables grouping of commands
 687 
 688    example:  
 689     'pre+[bkp-verify]_post' translates to
 690     'pre_and_groupIn_bkp_or_verify_groupOut_post'
 691 
 692 COMMANDS:
 693   usage      get usage help text
 694 
 695   and/or/groupIn/groupOut  
 696              pseudo commands used in batches (see SEPARATORS above)
 697 
 698   create     creates a configuration profile
 699   backup     backup with pre/post script execution (batch: [pre_bkp_post]),
 700               full (if full_if_older matches or no earlier backup is found)
 701               incremental (in all other cases)
 702   pre/post   execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
 703   bkp        as above but without executing pre/post scripts
 704   full       force full backup
 705   incr       force incremental backup
 706   list [<age>]  
 707              list all files in backup (as it was at <age>, default: now)
 708   status     prints backup sets and chains currently in repository
 709   verify [<age>] [--compare-data]  
 710              list files changed, since age if given
 711   verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]  
 712              list changes of a file or folder path in backup compared to a
 713              local path, since age if given
 714   restore <target_path> [<age>]  
 715              restore the complete backup to <target_path> [as it was at <age>]
 716   fetch <src_path> <target_path> [<age>]  
 717              fetch single file/folder from backup [as it was at <age>]
 718   purge [<max_age>] [--force]  
 719              list outdated backup files (older than \$MAX_AGE)
 720               [use --force to actually delete these files]
 721   purgeFull [<max_full_backups>] [--force]  
 722              list outdated backup files (\$MAX_FULL_BACKUPS being the number of
 723              full backups and associated incrementals to keep, counting in 
 724              reverse chronological order)
 725               [use --force to actually delete these files]
 726   purgeIncr [<max_fulls_with_incrs>] [--force]  
 727              list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being 
 728              the number of full backups which associated incrementals will be
 729              kept, counting in reverse chronological order) 
 730               [use --force to actually delete these files]
 731   cleanup [--force]  
 732              list broken backup chain files archives (e.g. after unfinished run)
 733               [use --force to actually delete these files]
 734 
 735   changelog  print changelog / todo list
 736   txt2man    feature for package maintainers - create a manpage based on the 
 737              usage output. download txt2man from http://mvertes.free.fr/, put 
 738              it in the PATH and run '$ME txt2man' to create a man page.
 739   version    show version information of $ME_NAME and needed programs
 740 
 741 OPTIONS:
 742   --force    passed to duplicity (see commands: 
 743              purge, purgeFull, purgeIncr, cleanup)
 744   --preview  do nothing but print out generated duplicity command lines
 745   --disable-encryption  
 746              disable encryption, overrides profile settings
 747 
 748 TIME FORMATS:
 749   For all time related parameters like age, max_age etc.
 750   Refer to the duplicity manpage for all available formats. Here some examples:
 751     2002-01-25T07:00:00+02:00 (full date time format string)
 752     2002/3/5 (date string YYYY/MM/DD)
 753     12D (interval, 12 days ago)
 754     1h78m (interval, 1 hour 78 minutes ago)
 755 
 756 PRE/POST SCRIPTS:
 757   Some useful internal duply variables are exported to the scripts.
 758 
 759     PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>, 
 760     GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
 761     CMD_<PREV|NEXT> (previous/next command), 
 762     CND_<PREV|NEXT> (condition before/after)
 763 
 764   The CMD_* variables were introduced to allow different actions according to 
 765   the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post' 
 766   will call the pre script two times, with CMD_NEXT variable set to 'bkp' 
 767   on the first and to 'verify' on the second run.
 768   CMD_ERR holds the exit code of the CMD_PREV .
 769 
 770 EXAMPLES:
 771   create profile 'humbug':  
 772     $ME humbug create (don't forget to edit this new conf file)
 773   backup 'humbug' now:  
 774     $ME humbug backup
 775   list available backup sets of profile 'humbug':  
 776     $ME humbug status
 777   list and delete outdated backups of 'humbug':  
 778     $ME humbug purge --force
 779   restore latest backup of 'humbug' to /mnt/restore:  
 780     $ME humbug restore /mnt/restore
 781   restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:  
 782     $ME humbug fetch etc/passwd /root/pw 4D
 783     (see "duplicity manpage", section TIME FORMATS)
 784   a one line batch job on 'humbug' for cron execution:  
 785     $ME humbug backup_verify_purge --force
 786   batch job to run a full backup with pre/post scripts:  
 787     $ME humbug pre_full_post
 788 
 789 FILES:
 790   in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
 791   conf             profile configuration file
 792   pre,post         pre/post scripts (see above for details)
 793   gpgkey.*.asc     exported GPG key files
 794   exclude          a globbing list of included or excluded files/folders
 795                    (see "duplicity manpage", section FILE SELECTION)
 796 
 797 $(hint_profile)
 798 
 799 SEE ALSO:
 800   duplicity man page:
 801     duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
 802 USAGE_EOF
 803 }
 804 
 805 # to check call 'duply txt2man | man -l -'
 806 function usage_txt2man {
 807   usage_info | \
 808   awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
 809   txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
 810 }
 811 
 812 function changelog {
 813   cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#(  )?/,"",$0);print}'
 814 }
 815 
 816 function create_config {
 817   if [ ! -d "$CONFDIR" ] ; then
 818     mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
 819   # create initial config file
 820     cat <<EOF >"$CONF"
 821 # gpg encryption settings, simple settings:
 822 #  GPG_KEY='disabled' - disables encryption alltogether
 823 #  GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
 824 #   sign if secret key of key1 is available use GPG_PW for sign & decrypt
 825 #  Note: you can specify keys via all methods described in gpg manpage,
 826 #        section "How to specify a user ID", escape commas (,) via backslash (\)
 827 #        e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
 828 #        as they are used to separate the entries
 829 #  GPG_PW='passphrase' - symmetric encryption using passphrase only
 830 GPG_KEY='${DEFAULT_GPG_KEY}'
 831 GPG_PW='${DEFAULT_GPG_PW}'
 832 # gpg encryption settings in detail (extended settings)
 833 #  the above settings translate to the following more specific settings
 834 #  GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
 835 #  GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
 836 #  GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
 837 #   encryption. If you want to deliver different passphrases for e.g. 
 838 #   several keys or symmetric encryption plus key signing you can use
 839 #   gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
 840 #   also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage 
 841 # notes on en/decryption
 842 #  private key and passphrase will only be needed for decryption or signing.
 843 #  decryption happens on restore and incrementals (compare archdir contents).
 844 #  for security reasons it makes sense to separate the signing key from the
 845 #  encryption keys. https://answers.launchpad.net/duplicity/+question/107216
 846 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
 847 #GPG_KEY_SIGN='<prvkey>'
 848 # set if signing key passphrase differs from encryption (key) passphrase
 849 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
 850 #GPG_PW_SIGN='<signpass>'
 851 
 852 # uncomment and set a file path or name force duply to use this gpg executable
 853 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
 854 #GPG='/usr/local/gpg-2.1/bin/gpg'
 855 
 856 # gpg options passed from duplicity to gpg process (default='')
 857 # e.g. "--trust-model pgp|classic|direct|always" 
 858 #   or "--compress-algo=bzip2 --bzip2-compress-level=9"
 859 #   or "--personal-cipher-preferences AES256,AES192,AES..."
 860 #   or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
 861 #   or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
 862 #      also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
 863 #GPG_OPTS=''
 864 
 865 # disable preliminary tests with the following setting
 866 #GPG_TEST='disabled'
 867 # disable automatic gpg key importing altogether
 868 #GPG_IMPORT='disabled'
 869 # disable automatic gpg key exporting to profile folder
 870 #GPG_EXPORT='disabled'
 871 
 872 # backend, credentials & location of the backup target (URL-Format)
 873 # generic syntax is
 874 #   scheme://[user[:password]@]host[:port]/[/]path
 875 # e.g.
 876 #   sftp://bob:secret@backupserver.com//home/bob/dupbkp
 877 # for details and available backends see duplicity manpage, section URL Format
 878 #   http://duplicity.nongnu.org/duplicity.1.html#sect7
 879 # BE AWARE:
 880 #   some backends (cloudfiles, S3 etc.) need additional env vars to be set to
 881 #   work properly, read after the TARGET definition for more details.
 882 # ATTENTION:
 883 #   characters other than A-Za-z0-9.-_.~ in the URL have to be
 884 #   replaced by their url encoded pendants, see
 885 #     http://en.wikipedia.org/wiki/Url_encoding
 886 #   if you define the credentials as TARGET_USER, TARGET_PASS below $ME
 887 #   will try to url_encode them for you if the need arises.
 888 TARGET='${DEFAULT_TARGET}'
 889 # optionally the username/password can be defined as extra variables
 890 # setting them here _and_ in TARGET results in an error
 891 # ATTENTION:
 892 #   there are backends that do not support the user/pass auth scheme.
 893 #   prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
 894 #   duplicity manpage. usually there is a NOTE section explaining if and which
 895 #   env vars should be set.
 896 #TARGET_USER='${DEFAULT_TARGET_USER}'
 897 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
 898 # e.g. for cloud files backend it might look like this (uncomment for use!)
 899 #export CLOUDFILES_USERNAME='someuser'
 900 #export CLOUDFILES_APIKEY='somekey'
 901 #export CLOUDFILES_AUTHURL ='someurl'
 902 # the following is an incomplete list (<backend>: comma separated env vars list)
 903 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
 904 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
 905 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
 906 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
 907 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
 908 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
 909 #        SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
 910 
 911 # base directory to backup
 912 SOURCE='${DEFAULT_SOURCE}'
 913 
 914 # a command that runs duplicity e.g. 
 915 #  shape bandwidth use via trickle
 916 #  "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
 917 #DUPL_PRECMD=""
 918 
 919 # override the used python interpreter, defaults to
 920 #  - parsed result of duplicity's shebang or 'python2'
 921 #   e.g. "python2" or "/usr/bin/python2.7"
 922 #PYTHON="python"
 923 
 924 # exclude folders containing exclusion file (since duplicity 0.5.14)
 925 # Uncomment the following two lines to enable this setting.
 926 #FILENAME='.duplicity-ignore'
 927 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
 928 
 929 # Time frame for old backups to keep, Used for the "purge" command.  
 930 # see duplicity man page, chapter TIME_FORMATS)
 931 #MAX_AGE=1M
 932 
 933 # Number of full backups to keep. Used for the "purgeFull" command. 
 934 # See duplicity man page, action "remove-all-but-n-full".
 935 #MAX_FULL_BACKUPS=1
 936 
 937 # Number of full backups for which incrementals will be kept for.
 938 # Used for the "purgeIncr" command.
 939 # See duplicity man page, action "remove-all-inc-of-but-n-full".
 940 #MAX_FULLS_WITH_INCRS=1
 941 
 942 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3) 
 943 # forces a full backup if last full backup reaches a specified age, for the 
 944 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
 945 # Uncomment the following two lines to enable this setting.
 946 #MAX_FULLBKP_AGE=1M
 947 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE " 
 948 
 949 # sets duplicity --volsize option (available since v0.4.3.RC7)
 950 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
 951 # VOLSIZE must be number of MB's to set the volume size to.
 952 # Uncomment the following two lines to enable this setting. 
 953 #VOLSIZE=50
 954 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
 955 
 956 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
 957 # default is 4, if not set
 958 #VERBOSITY=5
 959 
 960 # temporary file space. at least the size of the biggest file in backup
 961 # for a successful restoration process. (default is '/tmp', if not set)
 962 #TEMP_DIR=/tmp
 963 
 964 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
 965 # unencrypted meta data of the backup, enabling new incrementals without the 
 966 # need to decrypt backend metadata first. If empty or deleted somehow, the 
 967 # private key and it's password are needed.
 968 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
 969 #       big over time so you might want to put it not in the home dir.
 970 # default '~/.cache/duplicity/duply_<profile>/'
 971 # if set  '\${ARCH_DIR}/<profile>'
 972 #ARCH_DIR=/some/space/safe/.duply-cache
 973 
 974 # DEPRECATED setting
 975 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users 
 976 # to change the time separator from ':' to another character that will work 
 977 # on their system.  HINT: For Windows SMB shares, use --time-separator='_'.
 978 # NOTE: '-' is not valid as it conflicts with date separator.
 979 # ATTENTION: only use this with duplicity < 0.5.10, since then default file 
 980 #            naming is compatible and this option is pending depreciation 
 981 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
 982 
 983 # DEPRECATED setting
 984 # activates duplicity --short-filenames option, when uploading to a file
 985 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
 986 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
 987 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file 
 988 #            naming is compatible and this option is pending depreciation
 989 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
 990 
 991 # more duplicity command line options can be added in the following way
 992 # don't forget to leave a separating space char at the end
 993 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here " 
 994 
 995 EOF
 996 
 997 # create initial exclude file
 998     cat <<EOF >"$EXCLUDE"
 999 # although called exclude, this file is actually a globbing file list
1000 # duplicity accepts some globbing patterns, even including ones here
1001 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1002 # - dir/bar/foo
1003 # + dir/bar
1004 # - **
1005 # for more details see duplicity manpage, section File Selection
1006 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1007 
1008 EOF
1009 
1010   # Hints on first usage
1011   cat <<EOF
1012 
1013 Congratulations. You just created the profile '$FTPLCFG'.
1014 The initial config file has been created as 
1015 '$CONF'.
1016 You should now adjust this config file to your needs.
1017 
1018 $(hint_profile)
1019 
1020 EOF
1021 fi
1022 
1023 }
1024 
1025 # used in usage AND create_config
1026 function hint_profile {
1027   cat <<EOF
1028 IMPORTANT:
1029   Copy the _whole_ profile folder after the first backup to a safe place.
1030   It contains everything (duply related) needed to restore your backups. 
1031 
1032   Pay attention to (possibly later added) external files such as credentials
1033   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1034   (e.g. DPBX_ACCESS_TOKEN). 
1035   It is good policy to place those in the profile folder if possible at all.
1036     e.g. in case of 'multi://' target the config .json file
1037   Env vars should be added to duply profiles' conf file.
1038 
1039   Keep access to these files restricted as they contain information (gpg key,
1040   passphrases etc.) to access and modify your backups.
1041   
1042   Finally:
1043   You should attempt a restore from an unrelated host to be sure you really
1044   have everything needed for restoration.
1045 
1046   Repeat these steps after _all_ configuration changes. Some configuration 
1047   options are crucial for restoration.
1048 
1049 EOF
1050 }
1051 
1052 function separator {
1053   echo "--- $@ ---"
1054 }
1055 
1056 function inform {
1057   echo -e "\nINFO:\n\n$@\n"
1058 }
1059 
1060 function warning {
1061   echo -e "\nWARNING:\n\n$@\n"
1062 }
1063 
1064 function warning_oldhome {
1065   local old=$1 new=$2
1066   warning " ftplicity changed name to duply since you created your profiles.
1067   Please rename the old folder
1068   '$old'
1069   to
1070   '$new'
1071   and this warning will disappear.
1072   If you decide not to do so profiles will _only_ work from the old location."
1073 }
1074 
1075 function error_print {
1076   echo -e "$@" >&2
1077 }
1078 
1079 function error {
1080   error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1081   exit -1
1082 }
1083 
1084 function error_gpg {
1085   [ -n "$2" ] && local hint="\n  $2\n\n  "
1086   
1087   error "$1
1088 
1089 Hint${hint:+s}:
1090   ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1091   Don't forget the used _password_ as you will need it.
1092   When done enter the 8 digit id & the password in the profile conf file.
1093 
1094   The key id can be found doing a 'gpg --list-keys'. In the example output 
1095   below the key id for the public key would be FFFFFFFF.
1096 
1097   pub   1024D/FFFFFFFF 2007-12-17
1098   uid                  duplicity
1099   sub   2048g/899FE27F 2007-12-17
1100 "
1101 }
1102 
1103 function error_gpg_test {
1104   [ -n "$2" ] && local hint="\n  $2\n\n  "
1105 
1106   error "$1
1107 
1108 Hint${hint:+s}:
1109   ${hint}This error means that gpg is probably misconfigured or not working 
1110   correctly. The error message above should help to solve the problem.
1111   However, if for some reason $ME_NAME should misinterpret the situation you 
1112   can define GPG_TEST='disabled' in the conf file to bypass the test.
1113   Please do not forget to report the bug in order to resolve the problem
1114   in future versions of $ME_NAME.
1115 "
1116 }
1117 
1118 function error_path {
1119   error "$@
1120 PATH='$PATH'
1121 "
1122 }
1123 
1124 function error_to_string {
1125 	[ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1126 }
1127 
1128 function duplicity_version_get {
1129   # nothing to do, just print
1130   var_isset DUPL_VERSION && return
1131   
1132   local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1133   # only run with a user specific python if configured (running by default
1134   # breaks homebrew as they place a shell wrapper for duplicity in path)
1135   PYTHON_BIN="$(python_binary)" &&\
1136     CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
1137   CMD="$CMD --version 2>&1"
1138   DUPL_VERSION_OUT=$(eval "$CMD")
1139   DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1140   #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1141   DUPL_VERSION_VALUE=0
1142   DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1143   if (match(v,/[^\.0-9]+[0-9]*$/)){
1144     rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1145   if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1146   split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1147   print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1148   print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1149   print "DUPL_VERSION_RC=\047"rc"\047"
1150   print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1151   }')
1152   eval "$DUPL_VERSION_AWK"
1153   #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1154 
1155   # doublecheck findings and report error
1156   if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1157     inform "duplicity version check failed (please report, this is a bug)
1158 the command
1159   $CMD
1160 resulted in
1161   $DUPL_VERSION_OUT
1162 "
1163   elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1164     error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1165 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1166 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1167   fi
1168 }
1169 
1170 function duplicity_version_ge {
1171   [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1172 }
1173 
1174 function duplicity_version_lt {
1175   ! duplicity_version_ge "$1"
1176 }
1177 
1178 # parse interpreter from duplicity shebang
1179 function duplicity_python_binary_parse {
1180   # cached result
1181   ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
1182 
1183   # parse it or warn
1184   local DUPL_BIN=$(which duplicity)
1185   DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1186   if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1187     warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
1188 Will assume it is '$DEFAULT_PYTHON'."
1189     DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
1190   fi
1191 }
1192 
1193 function run_script { # run pre/post scripts
1194   local ERR=0
1195   local SCRIPT="$1"
1196   if [ ! -z "$PREVIEW" ] ; then	
1197     echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1198   elif [ -r "$SCRIPT" ] ; then 
1199     echo -n "Running '$SCRIPT' "
1200     if [ -x "$SCRIPT" ]; then
1201       OUT=$("$SCRIPT" 2>&1)
1202       ERR=$?
1203     else
1204       OUT=$(. "$SCRIPT" 2>&1)
1205       ERR=$?
1206     fi
1207     [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1208     echo -en ${OUT:+"Output: $OUT\n"} ;
1209   else
1210     echo "Skipping n/a script '$SCRIPT'."
1211   fi
1212   return $ERR
1213 }
1214 
1215 function run_cmd {
1216   # run or print escaped cmd string
1217   local CMD_ERR=0
1218   if [ -n "$PREVIEW" ]; then
1219     CMD_OUT=$( echo "$@ 2>&1" )
1220     CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1221   elif [ -n "$CMD_DISABLED" ]; then
1222     CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1223   else
1224     echo -n -e "$CMD_MSG"
1225     CMD_OUT=` eval "$@" 2>&1 `
1226     CMD_ERR=$?
1227     if [ "$CMD_ERR" = "0" ]; then
1228       CMD_MSG=" (OK)"
1229     else
1230       CMD_MSG=" (FAILED)"
1231     fi
1232   fi
1233   echo -e "$CMD_MSG"
1234   # reset
1235   unset CMD_DISABLED CMD_MSG
1236   return $CMD_ERR
1237 }
1238 
1239 function qw { quotewrap "$@"; }
1240 
1241 function quotewrap {
1242   local param="$@"
1243   # quote strings having non word chars (e.g. spaces)
1244   if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1245     echo "$param" | awk '{\
1246       gsub(/[\047]/,"\047\\\047\047",$0);\
1247       gsub(/[\042]/,"\047\\\042\047",$0);\
1248       print "\047"$0"\047"}'
1249     return
1250   fi
1251   echo $param
1252 }
1253 
1254 function duplicity_params_global {
1255   # already done? return
1256   var_isset 'DUPL_PARAMS_GLOBAL' && return
1257   local DUPL_ARG_ENC
1258 
1259   # use key only if set in config, else leave it to symmetric encryption
1260   if gpg_disabled; then
1261     local DUPL_PARAM_ENC='--no-encryption'
1262   else
1263     local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1264     gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1265     # interpret password settings
1266     var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1267     var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1268   fi
1269 
1270   local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1271 
1272   # set name for dupl archive folder, since 0.6.0
1273   if duplicity_version_ge 601; then
1274     local DUPL_ARCHDIR=''
1275     if var_isset 'ARCH_DIR'; then
1276       DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1277       # reuse erronously duply_ prefixed folders from bug #117
1278       if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1279         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1280       else
1281         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1282       fi
1283     else
1284       DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1285     fi
1286   fi
1287 
1288 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1289 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1290  ${GPG_OPTS}"
1291 
1292 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1293  ${DUPL_ARG_ENC}"
1294 }
1295 
1296 # function to filter the DUPL_PARAMS var from user conf
1297 function duplicity_params_conf {
1298   # reuse cmd var from main loop
1299   ## in/exclude parameters are currently not supported on restores
1300   if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1301     # filter exclude params from fetch/restore/status
1302     eval "stripXcludes $DUPL_PARAMS"
1303     return
1304   fi
1305 
1306   # nothing done, print unchanged
1307   echo "$DUPL_PARAMS"
1308 }
1309 
1310 # strip in/exclude parameters from param string
1311 function stripXcludes {
1312   local STRIPNEXT OUT;
1313   for p in "$@"; do
1314     if [ -n "$STRIPNEXT" ]; then
1315       unset STRIPNEXT
1316       # strip the value of previous parameter
1317       continue
1318     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1319       # strips e.g. --include /foo/bar
1320       STRIPNEXT="yes"
1321       continue
1322     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1323       # strips e.g. --include=/foo/bar
1324       continue
1325     fi
1326     
1327     OUT="$OUT $(qw "$p")"
1328   done
1329   echo "$OUT"
1330 }
1331 
1332 function duplify { # the actual wrapper function
1333   local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1334 
1335   # put command (with params) first in duplicity parameters
1336   for param in "$@" ; do
1337   # split cmd from params (everything before splitchar --)
1338     if [ "$param" == "--" ] ; then
1339       PARAMSNOW=1
1340     else
1341       # wrap in quotes to protect from spaces
1342       [ ! $PARAMSNOW ] && \
1343         DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1344       || \
1345         DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1346     fi
1347   done
1348 
1349   # init global duplicity parameters same for all tasks
1350   duplicity_params_global
1351 
1352   local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
1353   # run in cmd line preview mode if requested
1354   var_isset 'PREVIEW' && RUN=echo
1355   # try to resolve duplicity path for usage with python interpreter
1356   DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
1357   # only run with a user specific python if configured (running by default
1358   # breaks homebrew as they place a shell wrapper for duplicity in path)
1359   PYTHON_BIN="$(python_binary)" &&\
1360     BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
1361 
1362 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1363  ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1364  $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1365 
1366   local ERR=$?
1367   return $ERR
1368 }
1369 
1370 function secureconf { # secure the configuration dir
1371 	#PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1372 	local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1373 	if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1374 		chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1375 		warning "The profile's folder 
1376 '$CONFDIR'
1377 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1378 	fi
1379 }
1380 
1381 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1382 function date_fix {
1383 	local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1384 	local date
1385 	#[ "$1" == "%N" ] && return #test the no nsec test below
1386 	# gnu date with -d @epoch
1387 	date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1388 		echo $date && return
1389 	# date bsd,osx with -r epoch
1390 	date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1391 		echo $date && return	
1392 	# date busybox with -d epoch -D %s
1393 	date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1394 		echo $date && return
1395 	## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1396 	# python fallback
1397 	date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1398 		echo $date && return
1399 	# awk fallback
1400 	date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1401 		echo $date && return
1402 	# perl fallback
1403 	date=$(perl  -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1404 		echo $date && return
1405 	# error
1406 	echo "ERROR"
1407 	return 1
1408 }
1409 
1410 function nsecs {
1411 	local NSECS
1412 	# test if date supports nanosecond output
1413 	if ! var_isset NSECS_DISABLED; then
1414 		NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1415 		[ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1416 	fi
1417 
1418 	# add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1419 	if [ "$NSECS_DISABLED" == "1" ]; then
1420 		date_fix %s000000000
1421 	else
1422 		date_fix %s%N
1423 	fi
1424 }
1425 
1426 function nsecs_to_sec {
1427 	echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1428 }
1429 
1430 function datefull_from_nsecs {
1431 	date_from_nsecs $1 '%F %T'
1432 }
1433 
1434 function date_from_nsecs {
1435 	local FORMAT=${2:-%T}
1436 	local TIME=$(nsecs_to_sec $1)
1437 	local SECS=${TIME%.*}
1438 	local DATE=$(date_fix "%T" ${SECS:-0})
1439 	echo $DATE.${TIME#*.}
1440 }
1441 
1442 function var_isset {
1443   if [ -z "$1" ]; then
1444     echo "ERROR: function var_isset needs a string as parameter"
1445   elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1446     return 0
1447   fi
1448   return 1
1449 }
1450 
1451 function is_condition {
1452   local CMD=$(tolower "$1")
1453   [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1454 }
1455 
1456 function is_groupMarker {
1457   local CMD=$(tolower "$1")
1458   [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1459 }
1460 
1461 function is_command {
1462   local CMD=$(tolower "$1")
1463   ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1464 }
1465 
1466 function url_encode {
1467   # utilize python, silently do nothing on error - because no python no duplicity
1468   OUT=$("$(python_binary)" -c "
1469 try: import urllib.request as urllib
1470 except ImportError: import urllib
1471 print(urllib.${2}quote('$1'));
1472 " 2>/dev/null ); ERR=$?
1473   [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1474 }
1475 
1476 function url_decode {
1477   # reuse function above with a simple string param hack
1478   url_encode "$1" "un"
1479 }
1480 
1481 function toupper {
1482   echo "$@"|awk '$0=toupper($0)'
1483 }
1484 
1485 function tolower {
1486   echo "$@"|awk '$0=tolower($0)'
1487 }
1488 
1489 function isnumber {
1490   case "$*" in
1491     ''|*[!0-9]*) return 1;;
1492     *) return 0;;
1493   esac
1494 }
1495 
1496 #function tmp_space {
1497 #  
1498 #  if ! isnumber $VOLSIZE; then
1499 #    inform "failed to determine free space (please report, this is a bug)"
1500 #    return
1501 #  fi
1502 #  
1503 # get free temp space
1504 #  TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1505 #  # check for free space or FAIL
1506 #  if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1507 #    error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1508 #than one duplicity volume (${VOLSIZE}MB).
1509 #    
1510 #  Hint: Free space or change TEMP_DIR setting."
1511 #fi
1512 #
1513 #}
1514 
1515 function gpg_disabled {
1516   echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1517 }
1518 
1519 # usage: join SEPARATOR "entry1" "entry2"
1520 function join {
1521   local SEP="$1" ENTRY OUT; shift;
1522   for ENTRY in "$@"; do
1523     ENTRY=${ENTRY//$SEP/\\$SEP}
1524     [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1525   done
1526   echo $OUT
1527 }
1528 
1529 function gpg_testing {
1530   [ "$GPG_TEST" != "disabled" ]
1531 }
1532 
1533 function gpg_signing {
1534   echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1535 }
1536 
1537 function gpg_keytype {
1538   echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1539 }
1540 
1541 # parameter key id, key_type
1542 function gpg_keyfile {
1543   local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1544   local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1545   echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1546 }
1547 
1548 # parameter key id
1549 function gpg_import {
1550   local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1551   [ "$GPG_IMPORT" = "disabled" ] && {
1552     echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1553     return
1554   }
1555 
1556   # create a list of legacy key file names and current naming scheme
1557   # we always import pub and sec if they are avail in conf folder
1558   local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1559                    $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1560 
1561   # Try autoimport from existing old gpgkey files 
1562   # and new gpgkey.XXX.asc files (since v1.4.2)
1563   # and even newer gpgkey.XXX.[pub|sec].asc
1564   for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1565     FILE=${KEYFILES[$i]}
1566     if [ -f "$FILE" ]; then
1567       FOUND=1
1568       
1569       CMD_MSG="Import keyfile '$FILE' to keyring"
1570       run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1571       if [ "$?" != "0" ]; then 
1572         warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1573         ERR=1
1574         # continue with next
1575         continue
1576       fi
1577     fi
1578   done
1579 
1580   if [ "$FOUND" -eq 0 ]; then
1581     echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1582     return 1
1583   fi
1584 
1585   # try to set trust automagically
1586   CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1587   run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1588   if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then 
1589    # success on all levels, we're done
1590    return $ERR
1591   fi
1592 
1593   # failover: user has to set trust manually
1594   echo -e "For $ME_NAME to work you have to set the trust level 
1595 with the command \"trust\" to \"ultimate\" (5) now.
1596 Exit the edit mode of gpg with \"quit\"."
1597   CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1598   run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1599 
1600   return $ERR
1601 }
1602 
1603 # see 'How to specify a user ID' on gpg manpage
1604 function gpg_fingerprint {
1605   gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1606   awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1607 }
1608 
1609 function gpg_export_if_needed {
1610   [ "$GPG_EXPORT" = 'disabled' ] && { \
1611     echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1612     return
1613   }
1614 
1615   local SUCCESS FILE KEY_TYPE
1616   local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1617   for KEY_ID in "$@"; do
1618     # check if already exported, do it if not
1619     for KEY_TYPE in PUB SEC; do
1620       FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1621       if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1622 
1623         # exporting
1624         CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1625         # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1626         # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1627         run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1628         CMD_ERR=$?
1629 
1630         if [ "$CMD_ERR" = "0" ]; then
1631           CMD_MSG="Write file '"$(basename "$FILE")"'"
1632           run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1633         fi
1634 
1635         if [ "$CMD_ERR" != "0" ]; then
1636           warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1637         else
1638           SUCCESS=1
1639         fi
1640 
1641         # cleanup
1642         rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1643       fi
1644     done
1645   done
1646   
1647   [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1648 You should backup your changed profile folder now and store it in a safe place."
1649 }
1650 
1651 # replace all non-alnum chars with underscore (for file operations)
1652 function gpg_key_legalize {
1653   echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1654 }
1655 
1656 function gpg_key_cache {
1657   local RES
1658   local MODE=$1
1659   shift
1660   local PREFIX="GPG_KEY"
1661   local SUFFIX=$(gpg_key_legalize "$@")
1662   local KEYID="$*"
1663   local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1664   if [ "$MODE" = "RESET" ]; then
1665     eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1666     return 255
1667   elif ! var_isset "$CACHE"; then
1668     if [ "$MODE" = "PUB" ]; then
1669       RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1670     elif [ "$MODE" = "SEC" ]; then
1671       RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1672     else
1673       return 255
1674     fi
1675     eval $CACHE=$RES
1676   fi
1677   eval return \$$CACHE
1678 }
1679 
1680 function gpg_pub_avail {
1681   gpg_key_cache PUB "$@"
1682 }
1683 
1684 function gpg_sec_avail {
1685   gpg_key_cache SEC "$@"
1686 }
1687 
1688 function gpg_key_format {
1689   echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1690 }
1691 
1692 # splits a comma separated line into lines, respects escaped commas
1693 function gpg_split_keyset {
1694   local LIST
1695   LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1696   echo -e "$LIST"
1697 }
1698 
1699 function gpg_prefix_keyset {
1700   local PREFIX="$1" OUT=""
1701   shift
1702   for KEY_ID in "$@"; do
1703     OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1704   done
1705   echo $OUT
1706 }
1707 
1708 # grep a variable from conf text file (currently not used)
1709 function gpg_passwd {
1710   [ -r "$CONF" ] && \
1711   awk '/^[ \t]*GPG_PW[ \t=]/{\
1712         sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1713         gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1714         print $0; exit}' "$CONF"
1715 }
1716 
1717 # return success if at least one secret key is available
1718 function gpg_key_decryptable {
1719   local KEY_ID
1720   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1721     gpg_sec_avail "$KEY_ID" && return 0
1722   done
1723   return 1
1724 }
1725 
1726 function gpg_symmetric {
1727   [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1728 }
1729 
1730 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1731 function gpg_param_passwd {
1732   var_isset GPG_USEAGENT && exit 1
1733   
1734   if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1735     echo "--passphrase-fd 0 --batch"
1736   fi
1737 }
1738 
1739 # select the earliest defined and create an "echo <value> |" string
1740 function gpg_pass_pipein {
1741   var_isset GPG_USEAGENT && exit 1
1742   
1743   for var in "$@"
1744   do
1745     if var_isset "$var"; then
1746       echo "echo $(qw $(eval echo \$$var)) |"
1747       return 0
1748     fi
1749   done
1750   
1751   return 1
1752 }
1753 
1754 # checks if gpg-agent is available, returns error code
1755 # 0 on success
1756 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1757 # 2 if GPG_AGENT_INFO is stale
1758 # 3 cannot connect to gpg-agent
1759 function gpg_agent_avail {
1760   # GPG_AGENT_INFO is deprecated in gpg2.1, 
1761   # first try to connect to a possibly running agent here
1762   local ERR=3
1763   gpg-agent > /dev/null 2>&1 && return 0
1764 
1765   # detect stale pid in legacy GPG_AGENT_INFO env var
1766   if var_isset GPG_AGENT_INFO; then
1767     # check if a pid matching process is running at all
1768     local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1769     if isnumber "$GPG_AGENT_PID"; then
1770       ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1771     fi
1772   fi
1773 
1774   return $ERR
1775 }
1776 
1777 function gpg_custom_binary {
1778   var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1779     echo "--gpg-binary $(qw "$GPG")"
1780 }
1781 
1782 function gpg_binary {
1783   local BIN
1784   var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1785   echo "$BIN"
1786 }
1787 
1788 function gpg_avail {
1789   lookup "$(gpg_binary)"
1790 }
1791 
1792 # enforce the use of our selected gpg binary
1793 function gpg {
1794   command "$(gpg_binary)" "$@"
1795 }
1796 export -f gpg
1797 
1798 # start of script #######################################################################
1799 
1800 # confidentiality first, all we create is only readable by us
1801 umask 077
1802 
1803 # check if ftplicity is there & executable
1804 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1805 
1806 if [ ${#@} -eq 1 ]; then
1807   cmd="${1}"
1808 else
1809   FTPLCFG="${1}" ; cmd="${2}"
1810 fi
1811 
1812 # deal with command before profile validation calls
1813 # show requested version
1814 # OR requested usage info
1815 # OR create a profile
1816 # OR fall through
1817 ##if [ ${#@} -le 2 ]; then
1818 case "$cmd" in
1819   changelog)
1820     changelog
1821     exit 0
1822     ;;
1823   create)
1824     set_config
1825     if [ -d "$CONFDIR" ]; then
1826       error "The profile '$FTPLCFG' already exists in
1827 '$CONFDIR'.
1828 
1829 Hint:
1830  If you _really_ want to create a new profile by this name you will 
1831  have to manually delete the existing profile folder first."
1832       exit 1
1833     else
1834       create_config
1835       exit 0
1836     fi
1837     ;;
1838   txt2man)
1839     set_config
1840     usage_txt2man
1841     exit 0
1842     ;;
1843   usage|-help|--help|-h|-H)
1844     set_config
1845     usage_info
1846     exit 0
1847     ;;
1848   version|-version|--version|-v|-V)
1849     # profile can override GPG/PYTHON, so import it if it was given
1850     var_isset FTPLCFG && {
1851       set_config
1852       [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1853     }
1854     version_info_using
1855     exit 0
1856     ;;
1857   # fallthrough.. we got a command that needs an existing profile
1858   *)
1859     # if we reach here, user either forgot profile or chose wrong profileless command
1860     if [ ${#@} -le 1 ]; then
1861       error "\
1862  Missing or wrong parameters. 
1863  Only the commands 
1864    changelog, create, usage, txt2man, version
1865  can be called without selecting an existing profile first.
1866  Your command was '$cmd'.
1867 
1868  Hint: Run '$ME usage' to get help."
1869     fi
1870 esac
1871 
1872 
1873 # Hello world
1874 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1875 
1876 # check system environment
1877 
1878 # is duplicity avail
1879 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1880 # init, exec duplicity version check info
1881 duplicity_python_binary_parse
1882 duplicity_version_get
1883 
1884 # check for certain important helper programs
1885 for f in awk grep "$(python_binary)"; do
1886   lookup "$f" || \
1887     error_path "$f missing. installed und available in path?"
1888 done
1889 
1890 ### read configuration
1891 set_config
1892 # check validity
1893 if [ ! -d "$CONFDIR" ]; then 
1894     error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1895 '$CONFDIR'.
1896 
1897 Hints:
1898 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1899  Use '$ME <name> create' to create a new profile.
1900  Use '$ME usage' to get usage help."
1901 elif [ ! -x "$CONFDIR" ]; then
1902     error "\
1903 Profile folder in '$CONFDIR' cannot be accessed.
1904 
1905 Hint: 
1906  Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1907 elif [ ! -f "$CONF" ] ; then
1908   error "'$CONF' not found."
1909 elif [ ! -r "$CONF" ] ; then
1910   error "'$CONF' not readable."
1911 else
1912   . "$CONF"
1913   #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1914   TEMP_DIR=${TEMP_DIR:-'/tmp'}
1915   # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1916   if var_isset 'TARGET_PW'; then
1917     TARGET_PASS="${TARGET_PW}"
1918   fi
1919 fi
1920 echo "Using profile '$CONFDIR'."
1921 
1922 # secure config dir, if needed w/ warning
1923 secureconf
1924 
1925 # split TARGET in handy variables
1926 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1927   target=$0; match(target,/^([^\/:]+):\/\//); \
1928   prot=substr(target,RSTART,RLENGTH);\
1929   rest=substr(target,RSTART+RLENGTH); \
1930   if (credsavail=match(rest,/^[^@]*@/)){\
1931     creds=substr(rest,RSTART,RLENGTH-1);\
1932     credcount=split(creds,cred,":");\
1933     rest=substr(rest,RLENGTH+1);\
1934     # split creds with regexp\
1935     match(creds,/^([^:]+)/);\
1936     user=substr(creds,RSTART,RLENGTH);\
1937     pass=substr(creds,RSTART+1+RLENGTH);\
1938   };\
1939   # filter quotes or escape them\
1940   gsub(/[\047\042]/,"",prot);\
1941   gsub(/[\047\042]/,"",rest);\
1942   gsub(/[\047]/,"\047\\\047\047",creds);\
1943   print "TARGET_URL_PROT=\047"prot"\047\n"\
1944          "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1945          "TARGET_URL_CREDS=\047"creds"\047\n";\
1946    if(user){\
1947      gsub(/[\047]/,"\047\\\047\047",user);\
1948      print "TARGET_URL_USER=\047"user"\047\n"}\
1949    if(pass){\
1950      gsub(/[\047]/,"\047\\\047\047",pass);\
1951      print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1952   }')
1953 eval "${TARGET_SPLIT_URL}"
1954 
1955 # fetch commmand from parameters ########################################################
1956 # Hint: cmds is also used to check if authentification info sufficient in the next step 
1957 cmds="$2"; shift 2
1958 
1959 # complain if command(s) missing
1960 [ -z $cmds ] && error "  No command given.
1961 
1962   Hint: 
1963     Use '$ME usage' to get usage help."
1964 
1965 # process params
1966 for param in "$@"; do
1967   #echo !$param!
1968   case "$param" in
1969     # enable ftplicity preview mode
1970     '--preview')
1971       PREVIEW=1
1972       ;;
1973     # interpret duplicity disable encr switch
1974     '--disable-encryption')
1975       GPG_KEY='disabled'
1976       ;;
1977     *)
1978       if [ `echo "$param" | grep -e "^-"` ] || \
1979          [ `echo "$last_param" | grep -e "^-"` ] ; then
1980         # forward parameter[/option pairs] to duplicity
1981         dupl_opts["${#dupl_opts[@]}"]=${param}
1982       else
1983         # anything else must be a parameter (e.g. for fetch, ...)
1984         ftpl_pars["${#ftpl_pars[@]}"]=${param}
1985       fi
1986       last_param=${param}
1987       ;;
1988   esac
1989 done
1990 
1991 # plausibility check config - VARS & KEY ################################################
1992 # check if src, trg, trg pw
1993 # auth info sufficient 
1994 # gpg key, gpg pwd (might be empty) set in config
1995 # OR key in local gpg db
1996 # OR key can be imported from keyfile 
1997 # OR fail
1998 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
1999  error " Source Path (setting SOURCE) not set or still default value in conf file 
2000  '$CONF'."
2001 
2002 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2003  error " Backup Target (setting TARGET) not set or still default value in conf file 
2004  '$CONF'."
2005 
2006 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2007      [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2008  error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}') 
2009  are configured with different values. There can be only one.
2010  
2011  Hint: Remove conflicting setting."
2012 
2013 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2014      [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2015  error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}') 
2016  are configured with different values. There can be only one.
2017  
2018  Hint: Remove conflicting setting."
2019 fi
2020 
2021 # GPG config plausibility check1 (disabled check) #############################
2022 if gpg_disabled; then
2023   : # encryption disabled, all is well
2024 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2025   warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file 
2026 '$CONF'.
2027 Will disable encryption for duplicity now.
2028 
2029 Hint: 
2030  If you really want to use _no_ encryption you can disable this warning by 
2031  setting GPG_KEY='disabled' in conf file."
2032   GPG_KEY='disabled'
2033 fi
2034 
2035 # GPG availability check (now we know if gpg is really needed)#################
2036 if ! gpg_disabled; then 
2037   gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2038 fi
2039 
2040 # Output versions info ########################################################
2041 using_info
2042 
2043 # GPG create key settings, config check2 (needs gpg) ##########################
2044 if gpg_disabled; then
2045   : # the following tests are not necessary
2046 else
2047 
2048 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2049 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2050 
2051 # enc key still default?
2052 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then 
2053   error_gpg "Encryption Key GPG_KEY still default in conf file 
2054 '$CONF'."
2055 fi
2056 
2057 # create array of gpg encr keys, for further processing
2058 OIFS="$IFS" IFS=$'\n'
2059 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2060 IFS="$OIFS"
2061 
2062 # pw set? 
2063 # symmetric needs one, always
2064 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2065   ; then
2066   error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) 
2067 is empty/not set or still default value in conf file 
2068 '$CONF'."
2069 fi
2070 # this is a technicality, we can only pump one pass via pipe into gpg
2071 # but symmetric already always needs one for encryption
2072 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2073   [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2074   error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2075 '$CONF'.
2076 This is unfortunately impossible. For details see duplicity manpage, 
2077 section 'A Note On Symmetric Encryption And Signing'.
2078 
2079 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2080 Tip2: Use gpg-agent."
2081 fi
2082 
2083 # test - GPG KEY AVAILABILITY ##################################################
2084 
2085 # check gpg public keys availability, try import if needed
2086 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2087   KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2088   # test availability, try to import, retest
2089   if ! gpg_pub_avail "${KEY_ID}"; then
2090     echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2091     gpg_import "${KEY_ID}" PUB
2092     gpg_key_cache RESET "${KEY_ID}"
2093     gpg_pub_avail "${KEY_ID}" || { \
2094       gpg_testing && error_gpg \
2095       "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2096       "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2097   as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2098   If not you can put it there and $ME_NAME will autoimport it on the next run.
2099   Alternatively import it manually as the user you plan to run $ME_NAME with."
2100     }
2101   else
2102     echo "Public key '${KEY_ID}' found in keychain."
2103   fi
2104 done
2105 
2106 # check gpg encr secret encryption keys availability and fail
2107 # if none is available after a round of importing trials
2108 gpg_key_decryptable || \
2109 {
2110   echo "Missing secret keys for decryption in keychain."
2111   for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2112     KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2113     # test availability, try to import, retest
2114     if ! gpg_sec_avail "${KEY_ID}"; then
2115     echo "Try to import secret key '${KEY_ID}' from profile."
2116       gpg_import "${KEY_ID}" SEC
2117       gpg_key_cache RESET "${KEY_ID}"
2118     fi
2119   done
2120   gpg_key_decryptable || \
2121   {
2122     gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2123 has a secret key in the keychain. Decryption will be impossible!"
2124   }
2125 }
2126 
2127 # gpg secret sign key availability
2128 # if none set, autoset first encryption key as sign key
2129 if ! gpg_signing; then
2130   echo "Signing disabled per configuration."
2131 # try first key, if one set
2132 elif ! var_isset 'GPG_KEY_SIGN'; then
2133   KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2134   if [ -z "${KEY_ID}" ]; then
2135     echo "Signing disabled. No GPG_KEY entries in config."
2136     GPG_KEY_SIGN='disabled'
2137   else  
2138     # use avail OR try import OR fail
2139     if gpg_sec_avail "${KEY_ID}"; then
2140       GPG_KEY_SIGN="${KEY_ID}"
2141     else
2142       echo "Signing secret key '${KEY_ID}' not found."
2143       gpg_import "${KEY_ID}" SEC
2144       gpg_key_cache RESET "${KEY_ID}"
2145       if gpg_sec_avail "${KEY_ID}"; then
2146         GPG_KEY_SIGN="${KEY_ID}"
2147       fi
2148     fi
2149 
2150     # interpret sign key setting
2151     if var_isset 'GPG_KEY_SIGN'; then
2152       echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2153     else
2154       echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2155       GPG_KEY_SIGN='disabled'
2156     fi
2157   fi
2158 else
2159   KEY_ID="${GPG_KEY_SIGN}"
2160   if ! gpg_sec_avail "${KEY_ID}"; then
2161     inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2162     gpg_import "${KEY_ID}" SEC
2163     gpg_key_cache RESET "${KEY_ID}"
2164     gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2165   fi
2166 fi
2167 
2168 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2169 # key enc can deal without, but might profit from gpg-agent
2170 # if GPG_PW is not set alltogether
2171 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2172 if ! gpg_symmetric && \
2173    ( ! var_isset GPG_PW || \
2174      ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2175 
2176   if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2177     warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2178   elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2179     warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2180   elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2181     warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2182   else
2183     echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2184     GPG_USEAGENT="--use-agent"
2185   fi
2186 fi
2187 
2188 # end GPG config plausibility check2 
2189 fi
2190 
2191 # config plausibility check - SPACE ###########################################
2192 
2193 # is tmp is a folder and writable
2194 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2195 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2196 if [ "$?" != "0" ]; then
2197     error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2198 fi
2199 
2200 
2201 # get volsize, default duplicity volume size is 25MB since v0.5.07
2202 VOLSIZE=${VOLSIZE:-25}
2203 # double if asynch is on
2204 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2205 
2206 # TODO: check for enough (async= upload space and WARN only
2207 #       use function tmp_space 
2208 #echo TODO: reimplent tmp space check
2209 
2210 
2211 # test - GPG SANITY #####################################################################
2212 # if encryption is disabled, skip this whole section
2213 if gpg_disabled; then
2214   echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2215 elif ! gpg_testing; then 
2216   echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2217 else
2218 
2219 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2220 function cleanup_gpgtest { 
2221   echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2222   rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2223 }
2224 
2225 # signing enabled?
2226 if gpg_signing; then
2227   CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2228   CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2229 fi
2230 
2231 # using keys
2232 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2233 
2234   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2235     CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2236   done
2237   # check encrypting
2238   CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2239   run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2240   CMD_ERR=$?
2241 
2242   if [ "$CMD_ERR" != "0" ]; then 
2243     KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2244     [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2245   'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2246   can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2247     error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2248   fi
2249 
2250   # check decrypting
2251   CMD_MSG="Test - Decrypt"
2252   gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2253   run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2254   CMD_ERR=$?
2255 
2256   if [ "$CMD_ERR" != "0" ]; then 
2257     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2258   fi
2259 
2260 # symmetric only
2261 else
2262   # check encrypting
2263   CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2264   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2265   CMD_ERR=$?
2266   if [ "$CMD_ERR" != "0" ]; then 
2267     error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2268   fi
2269 
2270   # check decrypting
2271   CMD_MSG="Test - Decryption with passphrase"
2272   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2273   CMD_ERR=$?
2274   if [ "$CMD_ERR" != "0" ]; then 
2275     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2276   fi
2277 fi
2278 
2279 # compare original w/ decryptginal
2280 CMD_MSG="Test - Compare"
2281 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2282 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2283 CMD_ERR=$?
2284 if [ "$CMD_ERR" = "0" ]; then 
2285   cleanup_gpgtest
2286 else
2287   error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2288 fi
2289 
2290 fi # end disabled
2291 
2292 ## an empty line
2293 #echo
2294 
2295 # Exclude file is needed, create it if necessary
2296 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2297 
2298 # export only used keys, if bkp not already exists ######################################
2299 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2300 
2301 
2302 # command execution #####################################################################
2303 
2304 # urldecode url vars into plain text
2305 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2306 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2307 
2308 # defined TARGET_USER&PASS vars replace their URL pendants 
2309 # (double defs already dealt with)
2310 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2311 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2312 
2313 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2314 
2315 # issue some warnings
2316 case "$TARGET_URL_PROT_lowercase" in
2317   'cf+http')
2318     # info on missing AUTH_URL
2319     if ! var_isset 'CLOUDFILES_AUTHURL'; then
2320       inform "No CLOUDFILES_AUTHURL exported (in conf).
2321 Will use default which is probably rackspace."
2322     fi
2323     ;;
2324    'swift')
2325     # info on possibly missing AUTH_URL
2326        var_isset 'SWIFT_AUTHURL' ||\
2327        warning "\
2328 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2329     ;;
2330   'rsync')
2331     # everything in url (this backend does not support pass in env var)
2332     # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2333     # print warning older version is detected
2334     duplicity_version_lt 610 &&
2335       warning "\
2336 Duplicity version '$DUPL_VERSION' does not support providing the password as 
2337 env var for rsync backend. For security reasons you should consider to 
2338 update to a version greater than '0.6.10' of duplicity."
2339     ;;
2340 esac
2341 
2342 
2343 # for all protocols we put username in url and pass into env var 
2344 # for sec�rity reasons, we url_encode username to protect special chars
2345 # first sortout backends with special ways to handle password
2346 case "$TARGET_URL_PROT_lowercase" in
2347   'imap'|'imaps')
2348     var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2349     ;;
2350   *)
2351     # rest uses FTP_PASS var
2352     var_isset 'TARGET_URL_PASS' && \
2353       BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2354     ;;
2355 esac
2356 # insert url encoded username into target url if needed
2357 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2358   BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2359 else
2360   BACKEND_URL="$TARGET"
2361 fi
2362 
2363 
2364 # protect eval from special chars in url (e.g. open ')' in password, 
2365 # spaces in path, quotes) happens above in duplify() via quotewrap()
2366 SOURCE="$SOURCE"
2367 BACKEND_URL="$BACKEND_URL"
2368 EXCLUDE="$EXCLUDE"
2369 # since 0.7.03 --exclude-globbing-filelist is deprecated
2370 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist" 
2371 
2372 # translate backup to batch command 
2373 cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
2374 
2375 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2376 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2377   gsub(/\+/,\"_and_\",cmds);\
2378   gsub(/\-/,\"_or_\",cmds);\
2379   gsub(/\[/,\"_groupIn_\",cmds);\
2380   gsub(/\]/,\"_groupOut_\",cmds);\
2381   print cmds}")
2382 # convert cmds to array, lowercase for safety
2383 declare -a CMDS
2384 CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
2385 
2386 unset FTPL_ERR
2387 
2388 # run cmds
2389 for cmd in ${CMDS[*]};
2390 do
2391 
2392 ## init
2393 # raise index in cmd array for pre/post param
2394 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2395 
2396 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2397 
2398 # get next cmd,cnd vars
2399 nextno=$(( $CMD_NO ))
2400 while ! var_isset 'CMD_NEXT'
2401 do
2402   nextno=$(($nextno+1))
2403   if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2404     CMD_VALUE=${CMDS[$nextno]}
2405     is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2406     is_groupMarker "$CMD_VALUE" && continue
2407     CMD_NEXT="$CMD_VALUE"
2408   else
2409     CMD_NEXT='END'
2410   fi
2411 done
2412 
2413 # get prev cnd, cnds are skipped pseudocmds
2414 prevno=$(( $CMD_NO ));
2415 while ! var_isset 'CND_PREV'
2416 do
2417   prevno=$(($prevno-1))
2418   if [ "$prevno" -ge 0 ]; then
2419     CMD_VALUE=${CMDS[$prevno]}
2420     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2421     is_command "$CMD_VALUE" && break
2422   else
2423     break
2424   fi
2425 done
2426 
2427 # get prev cmd command minus skipped commands, only executed
2428 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2429 while ! var_isset 'CMD_PREV'
2430 do
2431   prevno=$(($prevno-1))
2432   if [ "$prevno" -ge 0 ]; then
2433     CMD_VALUE=${CMDS[$prevno]}
2434     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2435     is_groupMarker "$CMD_VALUE" && continue
2436     CMD_PREV="$CMD_VALUE"
2437   else
2438     CMD_PREV='START'
2439   fi
2440 done
2441 
2442 function get_cmd_skip_count {
2443   # find closing bracket, get group skip count
2444   local nextno=$CMD_NO
2445   local GRP_OPEN=0
2446   local GRP_SKIP=0
2447   local CMD_VALUE
2448   while [ "$nextno" -lt "${#CMDS[@]}" ]
2449   do
2450     nextno=$(($nextno+1))
2451     CMD_VALUE=${CMDS[$nextno]}
2452     GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2453     if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2454         break;
2455     elif [ "$CMD_VALUE" == 'groupin' ]; then
2456       GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2457     elif [ "$CMD_VALUE" == 'groupout' ]; then
2458       GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2459       if [ "$GRP_OPEN" -lt 1 ]; then
2460         break;
2461       fi
2462     fi
2463   done
2464 
2465   echo $GRP_SKIP;
2466 }
2467 
2468 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2469 unset SKIP_NOW
2470 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2471   # skip cnd/grp cmds silently
2472   is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2473   CMD_SKIP=$(($CMD_SKIP - 1))
2474   SKIP_NOW="yes"
2475 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2476   CMD_SKIP=$(get_cmd_skip_count)
2477   # incl. this "cmd"
2478   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2479   unset CMD_SKIPPED
2480   SKIP_NOW="yes"
2481 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2482   CMD_SKIP=$(get_cmd_skip_count)
2483   # incl. this "cmd"
2484   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2485   unset CMD_SKIPPED
2486   SKIP_NOW="yes"
2487 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2488   unset 'CMD_SKIP';
2489   SKIP_NOW="yes"
2490 fi
2491 
2492 # let's do the skip now
2493 if [ -n "$SKIP_NOW" ]; then
2494   # sum up how many commands we actually skipped for the prev var routines
2495   CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2496   continue
2497 fi
2498 
2499 # save start time
2500 RUN_START=$(nsecs)
2501 
2502 # export some useful env vars for external scripts/programs to use
2503 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2504        TARGET_URL_USER TARGET_URL_PASS \
2505        GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2506        GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2507        RUN_START
2508 
2509 # user info
2510 echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
2511 
2512 case "$(tolower $cmd)" in
2513   'pre'|'post')
2514     if [ "$cmd" == 'pre' ]; then
2515       script=$PRE
2516     else
2517       script=$POST
2518     fi
2519     # script execution in a subshell, protect us from failures/var overwrites
2520     ( run_script "$script" )
2521     ;;
2522   'bkp')
2523     duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2524           "$SOURCE" "$BACKEND_URL"
2525     ;;
2526   'incr')
2527     duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2528           "$SOURCE" "$BACKEND_URL"
2529     ;;
2530   'full')
2531     duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2532           "$SOURCE" "$BACKEND_URL"
2533     ;;
2534   'verify')
2535     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2536     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2537           "$BACKEND_URL" "$SOURCE"
2538     ;;
2539   'verifypath')
2540     TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2541     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2542     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2543   
2544   Hint: 
2545     Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2546 
2547     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2548           --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2549     ;;
2550   'list')
2551     # time param exists since 0.5.10+
2552     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2553     duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2554     ;;
2555   'cleanup')
2556     duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2557     ;;
2558   'purge')
2559     MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2560     [ -z "$MAX_AGE" ] && error "  Missing parameter <max_age>. Can be set in profile or as command line parameter."
2561     
2562     duplify remove-older-than "${MAX_AGE}" \
2563           -- "${dupl_opts[@]}" "$BACKEND_URL"
2564     ;;
2565   'purgefull')
2566     MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2567     [ -z "$MAX_FULL_BACKUPS" ] && error "  Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2568   
2569     duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2570           -- "${dupl_opts[@]}" "$BACKEND_URL"
2571     ;;
2572   'purgeincr')
2573     MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2574     [ -z "$MAX_FULLS_WITH_INCRS" ] && error "  Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2575   
2576     duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2577           -- "${dupl_opts[@]}" "$BACKEND_URL"
2578     ;;
2579   'restore')
2580     OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2581     [ -z "$OUT_PATH" ] && error "  Missing parameter target_path for restore.
2582   
2583   Hint: 
2584     Syntax is -> $ME <profile> restore <target_path> [<age>]"
2585     
2586     duplify  -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2587     ;;
2588   'fetch')
2589     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2590     TIME="${ftpl_pars[2]:-now}";
2591     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <src_path> or <target_path> for fetch.
2592   
2593   Hint: 
2594     Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2595     
2596     # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2597     duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2598               --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2599     ;;
2600   'status')
2601     duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2602     ;;    
2603   *)
2604     error "  Unknown command '$cmd'.
2605 
2606   Hint:
2607     Use '$ME usage' to get usage help."
2608     ;;
2609 esac
2610 
2611 CMD_ERR=$?
2612 RUN_END=$(nsecs)
2613 RUNTIME=$(( $RUN_END - $RUN_START ))
2614 
2615 # print message on error; set error code
2616 if [ "$CMD_ERR" -ne 0 ]; then
2617 	error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2618 	FTPL_ERR=1
2619 fi
2620 
2621 separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
2622 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2623 
2624 done
2625 
2626 exit ${FTPL_ERR}