Sourcecode / SVN
(Redirected from Duply-code)
Jump to navigation
Jump to search
Contents
duply sourcecode
duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.
SVN access
Browse the repository
https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/
SVN access
svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply
SVN Log
https://sourceforge.net/p/ftplicity/code/commit_browser
Latest Development Snapshot
mod time 2020-12-30
plain/text -> duply.sh
1 #!/usr/bin/env bash
2 #
3 ################################################################################
4 # duply (grown out of ftplicity), is a shell front end to duplicity that #
5 # simplifies the usage by managing settings for backup jobs in profiles. #
6 # It supports executing multiple commands in a batch mode to enable single #
7 # line cron entries and executes pre/post backup scripts. #
8 # Since version 1.5.0 all duplicity backends are supported. Hence the name #
9 # changed from ftplicity to duply. #
10 # See http://duply.net or http://ftplicity.sourceforge.net/ for more info. #
11 # (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany #
12 # (c) 2008-2020 Edgar Soldin (changes since version 1.3) #
13 ################################################################################
14 # LICENSE: #
15 # This program is licensed under GPLv2. #
16 # Please read the accompanying license information in gpl-2.0.txt. #
17 ################################################################################
18 # TODO/IDEAS/KNOWN PROBLEMS:
19 # - possibility to restore time frames (incl. deleted files)
20 # realizable by listing each backup and restore from
21 # oldest to the newest, problem: not performant
22 # - search file in all backups function and show available
23 # versions with backups date (list old avail since 0.6.06)
24 # - edit profile opens conf file in vi
25 # - implement log-fd interpretation
26 # - add a duplicity option check against the options pending
27 # deprecation since 0.5.10 namely --time-separator
28 # --short-filenames
29 # --old-filenames
30 # - add 'exclude_<command>' list usage e.g. exclude_verify
31 # - featreq 25: a download/install duplicity option
32 # - hint on install software if a piece is missing
33 # - import/export profile from/to .tgz function !!!
34 #
35 # CHANGELOG:
36 # 2.3 (30.12.2020)
37 # - don't import whole key pair anymore if only pub/sec is requested
38 # - gpg import routine informs on missing key files in profile now
39 # - add check/import needed secret key for decryption
40 # - featreq 50: Disable GPG key backups, implemented/added settings
41 # GPG_IMPORT/GPG_EXPORT='disabled' to conf template
42 #
43 # 2.2.2 (24.02.2020)
44 # - bugfix 120: Failures in "Autoset trust of key" during restore
45 # because of gpg2.2 fingerprint output change
46 #
47 # 2.2.1 (22.01.2020)
48 # - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
49 # - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
50 # - fix abbreviation spelling of 'e.g.'
51 #
52 # 2.2 (30.12.2018)
53 # - featreq 44: implement grouping for batch commands
54 # new separators are [] (square brackets) or groupIn/groupOut
55 # command 'backup' translates now to [pre_bkp_post] to be skipped as
56 # one block in case a condition was set in the batch instruction
57 #
58 # 2.1 (23.07.2018)
59 # - be more verbose when duplicity version detection fails
60 # - using info shows python binary's path for easier identification now
61 # - reworked python interpreter handling, it's either
62 # configured per PYTHON var
63 # unconfigured, parsed from duplicity shebang
64 # or set to current duplicity default 'python2' (was 'python' until now)
65 # - do not quotewrap strings because of slashes (e.g. paths) anymore
66 # - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
67 #
68 # 2.0.4 (20.02.2018)
69 # - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
70 # - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
71 # - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
72 # - bugfix debian 882159: duply: occasionally shows negative runtimes
73 #
74 # 2.0.3 (29.08.2017)
75 # - bugfix: "line 2231: CMDS: bad array subscript"
76 # - bugfix 112: "env: illegal option -- u" on MacOSX
77 #
78 # 2.0.2 (23.05.2017)
79 # - bugfix: never insert creds into file:// targets
80 # - bugfix: avail profiles hint sometimes shortend the names by one char
81 # - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
82 # - export condition before/after next/prev command as CND_PREV,CND_NEXT now
83 # - bugfix 97: Unknown command should be ERROR, not WARNING
84 #
85 # 2.0.1 (16.11.2016)
86 # - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
87 #
88 # 2.0 (27.10.2016)
89 # made this a major version change, as we broke backward compatibility anyway
90 # (see last change in v1.10). got complaints that rightfully pointed out
91 # that should only come w/ a major version change. so, here we go ;)
92 # if your backend stops working w/ this version create a new profile and
93 # export the env vars needed as described in the comments of the conf file
94 # directly above the SOURCE setting.
95 # Changes:
96 # - making sure multi spaces in TARGET survive awk processing
97 # - new env var PROFILE exported to scripts
98 # - fix 102: expose a unique timestamp variable for pre/post scripts
99 # actually a featreq. exporting RUN_START nanosec unix timestamp
100 # - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
101 # - fix 96: duply cannot handle two consecutive spaces in paths
102 #
103 # 1.11.3 (29.5.2016)
104 # - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
105 #
106 # 1.11.2 (11.2.2016)
107 # - fix "gpg: unsafe" version print out
108 # - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
109 # - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional
110 # documentation about needed env vars to template conf file
111 #
112 # 1.11.1 (18.12.2015)
113 # - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
114 # - reverted duply's default PYTHON to 'python'
115 #
116 # 1.11 (24.11.2015)
117 # - remove obsolete --ssh-askpass routine
118 # - add PYTHON conf var to allow global override of used python interpreter
119 # - enforced usage of "python2" in PATH as default interpreter for internal
120 # use _and_ to run duplicity (setup.py changed the shebang to the fixed
121 # path /usr/bin/python until 0.7.05, which we circumvent this way)
122 # - featreq 36: support gpg-connect-agent as a means to detect if an agent is
123 # running (thx Thomas Harning Jr.), used gpg-agent for detection though
124 # - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
125 # - key export routine respects gpg-agent usage now
126 #
127 # 1.10.1 (19.8.2015)
128 # - bugfix 86: Duply+Swift outputs warning
129 # - bugfix 87: Swift fails without BACKEND_URL
130 #
131 # 1.10 (31.7.2015)
132 # - featreq 37: busybox issues - fix awk, grep version detection,
133 # fix grep failure because --color=never switch is unsupported
134 # (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
135 # - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
136 # (thx Joachim Wiedorn, also for maintaining the debian package)
137 # - implemented base-/dirname as bash functions
138 # - featreq 31 " Support for duplicity Azure backend " - ignored a
139 # contributed patch by Scott McKenzie and instead opted for removing almost
140 # all code that deals with special env vars required by backends.
141 # adding and modifying these results in too much overhead so i dropped this
142 # feature. the future alternative for users is to consult the duplicity
143 # manpage and add the needed export definitions to the conf file.
144 # appended a commented example to the template conf below the auth section.
145 #
146 # 1.9.2 (21.6.2015)
147 # - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
148 # - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
149 # - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
150 # - added GPG conf var (see conf template for details)
151 # - added grep version output as it is an integral needed binary
152 # - added PYTHONPATH printout in version output
153 #
154 # 1.9.1 (13.10.2014)
155 # - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
156 # - bugfix: CMD_PREV contained command even if it was skipped
157 #
158 # 1.9.0 (24.8.2014)
159 # - bugfix: env vars were not exported when external script was executable
160 # - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.)
161 # see gpg manpage, section "How to specify a user ID"
162 # let gpg complain when the delivered values are invalid for whatever reason
163 # - started to rework tmp space checking, exposed folder & writable check
164 # TODO: reimplement enough file space available checking
165 #
166 # 1.8.0 (13.7.2014)
167 # - add command verifyPath to expose 'verify --file-to-restore' action
168 # - add time parameter support to verify command
169 # - add section time formats to usage output
170 #
171 # 1.7.4 (24.6.2014)
172 # - remove ubuntu one support, service is discontinued
173 # - featreq 31: add authenticated swift (contributed by Justus Seifert)
174 #
175 # 1.7.3 (3.4.2014)
176 # - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
177 #
178 # 1.7.2 (1.4.2014 "April,April")
179 # - bugfix: debian Bug#743190 "duply no longer allows restoration without
180 # gpg passphrase in conf file"
181 # GPG_AGENT_INFO env var is now needed to trigger --use-agent
182 # - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
183 #
184 # 1.7.1 (30.3.2014)
185 # - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to
186 # incompatibility with new minus batch separator
187 #
188 # 1.7.0 (20.3.2014)
189 # - disabled gpg key id plausibility check, too many valid possibilities
190 # - featreq 7 "Halt if precondition fails":
191 # added and(+), or(-) batch command(separator) support
192 # - featreq 26 "pre/post script with shebang line":
193 # if a script is flagged executable it's executed in a subshell
194 # now as opposed to sourced to bash, which is the default
195 # - bugfix: do not check if dpbx, swift credentials are set anymore
196 # - bugfix: properly escape profile name, archdir if used as arguments
197 # - add DUPL_PRECMD conf setting for use with e.g. trickle
198 #
199 # 1.6.0 (1.1.2014)
200 # - support gs backend
201 # - support dropbox backend
202 # - add gpg-agent support to gpg test routines
203 # - autoenable --use-agent if passwords were not defined in config
204 # - GPG_OPTS are now honored everywhere, keyrings or complete gpg
205 # homedir can thus be configured to be located anywhere
206 # - always import both secret and public key if avail from config profile
207 # - new explanatory comments in initial exclude file
208 # - bugfix 7: Duply only imports one key at a time
209 #
210 # 1.5.11 (19.07.2013)
211 # - purge-incr command for remove-all-inc-of-but-n-full feature added
212 # patch provided by Moritz Augsburger, thanks!
213 # - documented version command in man page
214 #
215 # 1.5.10 (26.03.2013)
216 # - minor indent and documentation fixes
217 # - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
218 # - bugfix: fix url_decoding generally and for python3
219 # - bugfix 3609075: wrong script results in status line (thx David Epping)
220 #
221 # 1.5.9 (22.11.2012)
222 # - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
223 # - restore/fetch now also ignores --include* or --exclude='foobar'
224 #
225 # 1.5.8 (26.10.2012)
226 # - bugfix 3575487: implement proper cloud files support
227 #
228 # 1.5.7 (10.06.2012)
229 # - bugfix 3531450: Cannot use space in target URL (file:///) anymore
230 #
231 # 1.5.6 (24.5.2012)
232 # - commands purge, purge-full have no default value anymore for security
233 # reasons; instead max value can be given via cmd line or must be set
234 # in profile; else an error is shown.
235 # - minor man page modifications
236 #
237 # versioning scheme will be simplified to [major].[minor].[patch] version
238 # with the next version raise
239 #
240 # 1.5.5.5 (4.2.2012)
241 # - bugfix 3479605: SEL context confused profile folder's permission check
242 # - colon ':' in url passphrase got ignored, added python driven url_decoding
243 # for user & pass to better process special chars
244 #
245 # 1.5.5.4 (16.10.2011)
246 # - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
247 # - add support for separate sign passphrase (needs duplicity 0.6.14+)
248 #
249 # 1.5.5.3 (1.10.2011)
250 # - bugfix 3416690: preview threw echo1 error
251 # - fix unknown cmds error usage & friends if more than 2 params were given
252 #
253 # 1.5.5.2 (23.9.2011)
254 # - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
255 # - bugfix: mawk does not support \W and did not split multikey definitions
256 # - all parameters should survive single (') and double (") quotes now
257 #
258 # 1.5.5.1 (7.6.2011)
259 # - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
260 # - bugfix 3312208: signing detection broke symmetric gpg test routine
261 #
262 # 1.5.5 (2.5.2011)
263 # - bugfix: fetch problem with space char in path, escape all params
264 # containing non word chars
265 # - list available profiles, if given profile cannot be found
266 # - added --use-agent configuration hint
267 # - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke
268 # fetch/restore
269 # - version command now prints out 'using installed' info
270 # - featreq 3166169: autotrust imported keys, based on code submitted by
271 # Martin Ellis - imported keys are now automagically trusted ultimately
272 # - new txt2man feature to create manpages for package maintainers
273 #
274 # 1.5.4.2 (6.1.2011)
275 # - new command changelog
276 # - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
277 # - bugfix: freebsd awk hangs on 'awk -W version'
278 # - bugfix 3150244: mawk does not know '--version'
279 # - minor help text improvements
280 # - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
281 #
282 # 1.5.4.1 (4.12.2010)
283 # - output awk, python, bash version now in prolog
284 # - shebang uses /usr/bin/env now for freebsd compatibility,
285 # bash not in /bin/bash
286 # - new --disable-encryption parameter,
287 # to override profile encr settings for one run
288 # - added exclude-if-present setting to conf template
289 # - bug 3126972: GPG_PW only needed for signing/symmetric encryption
290 # (even though duplicity still needs it)
291 #
292 # 1.5.4 (15.11.2010)
293 # - as of 1.5.3 already, new ARCH_DIR config option
294 # - multiple key support
295 # - ftplicity-Feature Requests-2994929: separate encryption and signing key
296 # - key signing of symmetric encryption possible (duplicity patch committed)
297 # - gpg tests disable switch
298 # - gpg tests now previewable and more intelligent
299 #
300 # 1.5.3 (1.11.2010)
301 # - bugfix 3056628: improve busybox compatibility, grep did not have -m param
302 # - bugfix 2995408: allow empty password for PGP key
303 # - bugfix 2996459: Duply erroneously escapes '-' symbol in username
304 # - url_encode function is now pythonized
305 # - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
306 # - feature 3059262: Make pre and post aware of parameters,
307 # internal parameters + CMD of pre or post
308 #
309 # 1.5.2.3 (16.4.2010)
310 # - bugfix: date again, should now work virtually anywhere
311 #
312 # 1.5.2.2 (3.4.2010)
313 # - minor bugfix: duplicity 0.6.8b version string now parsable
314 # - added INSTALL.txt
315 #
316 # 1.5.2.1 (23.3.2010)
317 # - bugfix: date formatting is awked now and should work on all platforms
318 #
319 # 1.5.2 (2.3.2010)
320 # - bugfix: errors print to STD_ERR now, failed tasks print an error message
321 # - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
322 # - simplified & cleaned profileless commands, removed second instance
323 # - generalized separator time routines
324 # - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
325 # - minor fixes
326 #
327 # 1.5.1.5 (5.2.2010)
328 # - bugfix: added special handling of credentials for rsync, imap(s)
329 #
330 # 1.5.1.4 (7.1.2010)
331 # - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
332 # - check if ncftp binary is available if url protocol is ftp
333 # - bugfix: duplicity output is now printed to screen directly to resolve
334 # 'mem alloc problem' bug report
335 # - bugfix: passwords will not be in the url anymore to solve the 'duply shows
336 # sensitive data in process listing' bug report
337 #
338 # 1.5.1.3 (24.12.2009) 'merry xmas'
339 # - bugfix: gpg pass now apostrophed to allow space and friends
340 # - bugfix: credentials are now url encoded to allow special chars in them
341 # a note about url encoding has been added to the conf template
342 #
343 # 1.5.1.2 (1.11.2009)
344 # - bugfix: open parenthesis in password broke duplicity execution
345 # - bugfix: ssh/scp backend does not always need credentials e.g. key auth
346 #
347 # 1.5.1.1 (21.09.2009)
348 # - bugfix: fixed s3[+http] TARGET_PASS not needed routine
349 # - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
350 # see https://sourceforge.net/tracker/index.php?func=detail&
351 # aid=2864410&group_id=217745&atid=1041147
352 #
353 # 1.5.1 (21.09.2009) - duply (fka. ftplicity)
354 # - first things first: ftplicity (being able to support all backends since
355 # some time) will be called duply (fka. ftplicity) from now on. The addendum
356 # is for the time being to circumvent confusion.
357 # - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity
358 # command failed
359 # - s3[+http] now supported natively by translating user/pass to access_key/
360 # secret_key environment variables needed by duplicity s3 boto backend
361 # - bugfix: additional output lines do not confuse version check anymore
362 # - list command supports now age parameter (patch by stefan on feature
363 # request tracker)
364 # - bugfix: option/param pairs are now correctly passed on to duplicity
365 # - bugfix: s3[+http] needs no TARGET_PASS if command is read only
366 #
367 # 1.5.0.2 (31.07.1009)
368 # - bugfix: insert password in target url didn't work with debian mawk
369 # related to previous bug report
370 #
371 # 1.5.0.1 (23.07.2009)
372 # - bugfix: gawk gensub dependency raised an error on debian's default mawk
373 # replaced with match/substr command combination (bug report)
374 # https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
375 # group_id=217745
376 #
377 # 1.5.0 (01.07.2009)
378 # - removed ftp limitation, all duplicity backends should work now
379 # - bugfix: date for separator failed on openwrt busybox date, added a
380 # detecting workaround, milliseconds are not available w/ busybox date
381 #
382 # 1.4.2.1 (14.05.2009)
383 # - bugfix: free temp space detection failed with lvm, fixed awk parse routine
384 #
385 # 1.4.2 (22.04.2009)
386 # - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
387 # armored ascii nature, the id helps if the key is switched for some reason
388 # im/export routines are updated accordingly (import is backward compatible
389 # to the old profile/gpgkey files)
390 # - profile argument is treated as path if it contains slashes
391 # (for details see usage)
392 # - non-ftplicity options (all but --preview currently) are now passed
393 # on to duplicity
394 # - removed need for stat in secure_conf, it is ls based now
395 # - added profile folder readable check
396 # - added gpg version & home info output
397 # - awk utility availability is now checked, because it was mandatory already
398 # - tmp space is now checked on writability and space requirement
399 # test fails on less than 25MB or configured $VOLSIZE,
400 # test warns if there is less than two times $VOLSIZE because
401 # that's required for --asynchronous-upload option
402 # - gpg functionality is tested now before executing duplicity
403 # test drive contains encryption, decryption, comparison, cleanup
404 # this is meant to detect non trusted or other gpg errors early
405 # - added possibility of doing symmetric encryption with duplicity
406 # set GPG_KEY="" or simply comment it out
407 # - added hints in config template on the depreciation of
408 # --short-filenames, --time-separator duplicity options
409 #
410 # new versioning scheme 1.4.2b => 1.4.2,
411 # beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
412 # to the first bug fixing version and 1.4.2.2 to the second and so on
413 # also the releases will now have a release date formatted (Day.Month.Year)
414 #
415 # 1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
416 # named exactly as the profile if existing in executing dir
417 # - improved plausibility checking of config and profile folder
418 # - secure_conf only acts if needed and prints a warning now
419 #
420 # 1.4.1b - introduce status (duplicity collection-status) command
421 # - pre/post script output printed always now, not only on errors
422 # - new config parameter GPG_OPTS to pass gpg options
423 # added examples & comments to profile template conf
424 # - reworked separator times, added duration display
425 # - added --preview switch, to preview generated command lines
426 # - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
427 # profiles because they have reasonable defaults now if not set
428 #
429 # 1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
430 # therefore backup translates to pre_bkp_post now
431 # - bugfix: new command bkp, which represents duplicity's
432 # default action (incr or full if full_if_older matches
433 # or no earlier backup chain is found)
434 #
435 # new versioning scheme 1.4 => 1.4.0, added new minor revision number
436 # this is meant to slow down the rapid version growing but still keep
437 # versions cleanly separated.
438 # only additional features will raise the new minor revision number.
439 # all releases start as beta, each bugfix release will raise the beta
440 # count, usually new features arrive before a version 'ripes' to stable
441 #
442 # 1.4.0b
443 # 1.4b - added startup info on version, time, selected profile
444 # - added time output to separation lines
445 # - introduced: command purge-full implements duplicity's
446 # remove-all-but-n-full functionality (patch by unknown),
447 # uses config variable $MAX_FULL_BACKUPS (default = 1)
448 # - purge config var $MAX_AGE defaults to 1M (month) now
449 # - command full does not execute pre/post anymore
450 # use batch command pre_full_post if needed
451 # - introduced batch mode cmd1_cmd2_etc
452 # (in turn removed the bvp command)
453 # - unknown/undefined command issues a warning/error now
454 # - bugfix: version check works with 0.4.2 and older now
455 # 1.3b3 - introduced pre/post commands to execute/debug scripts
456 # - introduced bvp (backup, verify, purge)
457 # - bugfix: removed need for awk gensub, now mawk compatible
458 # 1.3b2 - removed pre/post need executable bit set
459 # - profiles now under ~/.ftplicity as folders
460 # - root can keep profiles in /etc/ftplicity, folder must be
461 # created by hand, existing profiles must be moved there
462 # - removed ftplicity in path requirement
463 # - bugfix: bash < v.3 did not know '=~'
464 # - bugfix: purge works again
465 # 1.3 - introduces multiple profiles support
466 # - modified some script errors/docs
467 # - reordered gpg key check import routine
468 # - added 'gpg key id not set' check
469 # - added error_gpg (adds how to setup gpg key howto)
470 # - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
471 # - duplicity_version_check routine introduced
472 # - added time separator, shortnames, volsize, full_if_older
473 # duplicity options to config file (inspired by stevie
474 # from http://weareroot.de)
475 # 1.1.1 - bugfix: encryption reactivated
476 # 1.1 - introduced config directory
477 # 1.0 - first release
478 ################################################################################
479
480 # utility functions overriding binaries
481
482 # wrap grep to override possible env set GREP_OPTIONS=--color=always
483 function grep {
484 command env "GREP_OPTIONS=" grep "$@"
485 }
486
487 # implement basename in plain bash
488 function basename {
489 local stripped="${1%/}"
490 echo "${stripped##*/}"
491 }
492
493 # implement dirname in plain bash
494 function dirname {
495 echo ${1%/*}
496 }
497
498 # implement basic which in plain bash
499 function which {
500 type -p "$@"
501 }
502
503 # check availability of executables via file name or file paths
504 function lookup {
505 local bin="$1"
506 # look for file names in path via bash hash OR
507 # look for executables at given relative/absolute location
508 ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
509 }
510
511 # the python binary to use, exit code 0 when configured, else 1
512 function python_binary {
513 # if unset, parse from duplicity shebang
514 if ! var_isset 'PYTHON'; then
515 duplicity_python_binary_parse;
516 echo $DUPL_PYTHON_BIN;
517 return 1;
518 else
519 # tell if PYTHON was configured manually
520 echo $PYTHON;
521 return 0
522 fi
523 }
524
525 # important definitions #######################################################
526
527 ME_LONG="$0"
528 ME="$(basename $0)"
529 ME_NAME="${ME%%.*}"
530 ME_VERSION="2.3"
531 ME_WEBSITE="http://duply.net"
532
533 # default config values
534 DEFAULT_SOURCE='/path/of/source'
535 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
536 DEFAULT_TARGET_USER='_backend_username_'
537 DEFAULT_TARGET_PASS='_backend_password_'
538 DEFAULT_GPG='gpg'
539 DEFAULT_GPG_KEY='_KEY_ID_'
540 DEFAULT_GPG_PW='_GPG_PASSWORD_'
541 DEFAULT_PYTHON='python2'
542
543 # function definitions ##########################
544
545 function set_config { # sets global config vars
546 local CONFHOME_COMPAT="$HOME/.ftplicity"
547 local CONFHOME="$HOME/.duply"
548 local CONFHOME_ETC_COMPAT="/etc/ftplicity"
549 local CONFHOME_ETC="/etc/duply"
550
551 # confdir can be delivered as path (must contain /)
552 if [ `echo $FTPLCFG | grep /` ] ; then
553 CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
554 ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
555 && echo $(pwd)/${FTPLCFG} ) || \
556 echo ${FTPLCFG})
557 # or DEFAULT in home/.duply folder (NEW)
558 elif [ -d "${CONFHOME}" ]; then
559 CONFDIR="${CONFHOME}/${FTPLCFG}"
560 # or in home/.ftplicity folder (OLD)
561 elif [ -d "${CONFHOME_COMPAT}" ]; then
562 CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
563 warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
564 # root can put profiles under /etc/duply (NEW) if path exists
565 elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
566 CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
567 # root can keep profiles under /etc/ftplicity (OLD) if path exists
568 elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
569 CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
570 warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
571 # hmm no profile folder there, then use default for error later
572 else
573 CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
574 fi
575
576 # remove trailing slash, get profile name etc.
577 CONFDIR="${CONFDIR%/}"
578 PROFILE="${CONFDIR##*/}"
579 CONF="$CONFDIR/conf"
580 PRE="$CONFDIR/pre"
581 POST="$CONFDIR/post"
582 EXCLUDE="$CONFDIR/exclude"
583 KEYFILE="$CONFDIR/gpgkey.asc"
584 }
585
586 function version_info { # print version information
587 cat <<END
588 $ME_NAME version $ME_VERSION
589 ($ME_WEBSITE)
590 END
591 }
592
593 function version_info_using {
594 cat <<END
595 $(version_info)
596
597 $(using_info)
598 END
599 }
600
601 function using_info {
602 # init needed vars into global name space
603 lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
604 local NOTFOUND="INVALID"
605 local AWK_VERSION GREP_VERSION PYTHON_RUNNER
606 # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
607 AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
608 GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
609 PYTHON_RUNNER=$(python_binary)
610 local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
611 local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
612 local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
613 # print out
614 echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
615 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
616 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
617 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
618 }
619
620 function usage_info { # print usage information
621
622 cat <<USAGE_EOF
623 VERSION:
624 $(version_info)
625
626 DESCRIPTION:
627 Duply deals as a wrapper for the mighty duplicity magic.
628 It simplifies running duplicity with cron or on command line by:
629
630 - keeping recurring settings in profiles per backup job
631 - enabling batch operations e.g. backup_verify+purge
632 - executing pre/post scripts (different actions possible
633 depending on previous or next command or it's exit status)
634 - precondition checking for flawless duplicity operation
635
636 For each backup job one configuration profile must be created.
637 The profile folder will be stored under '~/.${ME_NAME}/<profile>'
638 (where ~ is the current users home directory).
639 Hint:
640 If the folder '/etc/${ME_NAME}' exists, the profiles for the super
641 user root will be searched & created there.
642
643 USAGE:
644 first time usage (profile creation):
645 $ME <profile> create
646
647 general usage in single or batch mode (see EXAMPLES):
648 $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
649
650 For batches the conditional separators can also be written as pseudo commands
651 and(+), or(-). See SEPARATORS for details.
652
653 Non $ME options are passed on to duplicity (see OPTIONS).
654 All conf parameters can also be defined in the environment instead.
655
656 PROFILE:
657 Indicated by a path or a profile name (<profile>), which is resolved
658 to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
659
660 Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
661 the folder manually before running $ME_NAME as superuser.
662 Note:
663 Already existing profiles in root's home folder will cease to work
664 unless they are moved to the new location manually.
665
666 example 1: $ME humbug backup
667
668 Alternatively a _path_ might be used e.g. useful for quick testing,
669 restoring or exotic locations. Shell expansion should work as usual.
670 Hint:
671 The path must contain at least one path separator '/',
672 e.g. './test' instead of only 'test'.
673
674 example 2: $ME ~/.${ME_NAME}/humbug backup
675
676 SEPARATORS:
677 _ (underscore)
678 neutral separator
679 + (plus sign), _and_
680 conditional AND
681 the next command will only be executed if the previous succeeded
682 - (minus sign), _or_
683 conditional OR
684 the next command will only be executed if the previous failed
685 [] (square brackets), _groupIn_/_groupOut_
686 enables grouping of commands
687
688 example:
689 'pre+[bkp-verify]_post' translates to
690 'pre_and_groupIn_bkp_or_verify_groupOut_post'
691
692 COMMANDS:
693 usage get usage help text
694
695 and/or/groupIn/groupOut
696 pseudo commands used in batches (see SEPARATORS above)
697
698 create creates a configuration profile
699 backup backup with pre/post script execution (batch: [pre_bkp_post]),
700 full (if full_if_older matches or no earlier backup is found)
701 incremental (in all other cases)
702 pre/post execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
703 bkp as above but without executing pre/post scripts
704 full force full backup
705 incr force incremental backup
706 list [<age>]
707 list all files in backup (as it was at <age>, default: now)
708 status prints backup sets and chains currently in repository
709 verify [<age>] [--compare-data]
710 list files changed, since age if given
711 verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]
712 list changes of a file or folder path in backup compared to a
713 local path, since age if given
714 restore <target_path> [<age>]
715 restore the complete backup to <target_path> [as it was at <age>]
716 fetch <src_path> <target_path> [<age>]
717 fetch single file/folder from backup [as it was at <age>]
718 purge [<max_age>] [--force]
719 list outdated backup files (older than \$MAX_AGE)
720 [use --force to actually delete these files]
721 purgeFull [<max_full_backups>] [--force]
722 list outdated backup files (\$MAX_FULL_BACKUPS being the number of
723 full backups and associated incrementals to keep, counting in
724 reverse chronological order)
725 [use --force to actually delete these files]
726 purgeIncr [<max_fulls_with_incrs>] [--force]
727 list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being
728 the number of full backups which associated incrementals will be
729 kept, counting in reverse chronological order)
730 [use --force to actually delete these files]
731 cleanup [--force]
732 list broken backup chain files archives (e.g. after unfinished run)
733 [use --force to actually delete these files]
734
735 changelog print changelog / todo list
736 txt2man feature for package maintainers - create a manpage based on the
737 usage output. download txt2man from http://mvertes.free.fr/, put
738 it in the PATH and run '$ME txt2man' to create a man page.
739 version show version information of $ME_NAME and needed programs
740
741 OPTIONS:
742 --force passed to duplicity (see commands:
743 purge, purgeFull, purgeIncr, cleanup)
744 --preview do nothing but print out generated duplicity command lines
745 --disable-encryption
746 disable encryption, overrides profile settings
747
748 TIME FORMATS:
749 For all time related parameters like age, max_age etc.
750 Refer to the duplicity manpage for all available formats. Here some examples:
751 2002-01-25T07:00:00+02:00 (full date time format string)
752 2002/3/5 (date string YYYY/MM/DD)
753 12D (interval, 12 days ago)
754 1h78m (interval, 1 hour 78 minutes ago)
755
756 PRE/POST SCRIPTS:
757 Some useful internal duply variables are exported to the scripts.
758
759 PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>,
760 GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
761 CMD_<PREV|NEXT> (previous/next command),
762 CND_<PREV|NEXT> (condition before/after)
763
764 The CMD_* variables were introduced to allow different actions according to
765 the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post'
766 will call the pre script two times, with CMD_NEXT variable set to 'bkp'
767 on the first and to 'verify' on the second run.
768 CMD_ERR holds the exit code of the CMD_PREV .
769
770 EXAMPLES:
771 create profile 'humbug':
772 $ME humbug create (don't forget to edit this new conf file)
773 backup 'humbug' now:
774 $ME humbug backup
775 list available backup sets of profile 'humbug':
776 $ME humbug status
777 list and delete outdated backups of 'humbug':
778 $ME humbug purge --force
779 restore latest backup of 'humbug' to /mnt/restore:
780 $ME humbug restore /mnt/restore
781 restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:
782 $ME humbug fetch etc/passwd /root/pw 4D
783 (see "duplicity manpage", section TIME FORMATS)
784 a one line batch job on 'humbug' for cron execution:
785 $ME humbug backup_verify_purge --force
786 batch job to run a full backup with pre/post scripts:
787 $ME humbug pre_full_post
788
789 FILES:
790 in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
791 conf profile configuration file
792 pre,post pre/post scripts (see above for details)
793 gpgkey.*.asc exported GPG key files
794 exclude a globbing list of included or excluded files/folders
795 (see "duplicity manpage", section FILE SELECTION)
796
797 $(hint_profile)
798
799 SEE ALSO:
800 duplicity man page:
801 duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
802 USAGE_EOF
803 }
804
805 # to check call 'duply txt2man | man -l -'
806 function usage_txt2man {
807 usage_info | \
808 awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
809 txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
810 }
811
812 function changelog {
813 cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#( )?/,"",$0);print}'
814 }
815
816 function create_config {
817 if [ ! -d "$CONFDIR" ] ; then
818 mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
819 # create initial config file
820 cat <<EOF >"$CONF"
821 # gpg encryption settings, simple settings:
822 # GPG_KEY='disabled' - disables encryption alltogether
823 # GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
824 # sign if secret key of key1 is available use GPG_PW for sign & decrypt
825 # Note: you can specify keys via all methods described in gpg manpage,
826 # section "How to specify a user ID", escape commas (,) via backslash (\)
827 # e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
828 # as they are used to separate the entries
829 # GPG_PW='passphrase' - symmetric encryption using passphrase only
830 GPG_KEY='${DEFAULT_GPG_KEY}'
831 GPG_PW='${DEFAULT_GPG_PW}'
832 # gpg encryption settings in detail (extended settings)
833 # the above settings translate to the following more specific settings
834 # GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
835 # GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
836 # GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
837 # encryption. If you want to deliver different passphrases for e.g.
838 # several keys or symmetric encryption plus key signing you can use
839 # gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
840 # also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage
841 # notes on en/decryption
842 # private key and passphrase will only be needed for decryption or signing.
843 # decryption happens on restore and incrementals (compare archdir contents).
844 # for security reasons it makes sense to separate the signing key from the
845 # encryption keys. https://answers.launchpad.net/duplicity/+question/107216
846 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
847 #GPG_KEY_SIGN='<prvkey>'
848 # set if signing key passphrase differs from encryption (key) passphrase
849 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
850 #GPG_PW_SIGN='<signpass>'
851
852 # uncomment and set a file path or name force duply to use this gpg executable
853 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
854 #GPG='/usr/local/gpg-2.1/bin/gpg'
855
856 # gpg options passed from duplicity to gpg process (default='')
857 # e.g. "--trust-model pgp|classic|direct|always"
858 # or "--compress-algo=bzip2 --bzip2-compress-level=9"
859 # or "--personal-cipher-preferences AES256,AES192,AES..."
860 # or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
861 # or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
862 # also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
863 #GPG_OPTS=''
864
865 # disable preliminary tests with the following setting
866 #GPG_TEST='disabled'
867 # disable automatic gpg key importing altogether
868 #GPG_IMPORT='disabled'
869 # disable automatic gpg key exporting to profile folder
870 #GPG_EXPORT='disabled'
871
872 # backend, credentials & location of the backup target (URL-Format)
873 # generic syntax is
874 # scheme://[user[:password]@]host[:port]/[/]path
875 # e.g.
876 # sftp://bob:secret@backupserver.com//home/bob/dupbkp
877 # for details and available backends see duplicity manpage, section URL Format
878 # http://duplicity.nongnu.org/duplicity.1.html#sect7
879 # BE AWARE:
880 # some backends (cloudfiles, S3 etc.) need additional env vars to be set to
881 # work properly, read after the TARGET definition for more details.
882 # ATTENTION:
883 # characters other than A-Za-z0-9.-_.~ in the URL have to be
884 # replaced by their url encoded pendants, see
885 # http://en.wikipedia.org/wiki/Url_encoding
886 # if you define the credentials as TARGET_USER, TARGET_PASS below $ME
887 # will try to url_encode them for you if the need arises.
888 TARGET='${DEFAULT_TARGET}'
889 # optionally the username/password can be defined as extra variables
890 # setting them here _and_ in TARGET results in an error
891 # ATTENTION:
892 # there are backends that do not support the user/pass auth scheme.
893 # prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
894 # duplicity manpage. usually there is a NOTE section explaining if and which
895 # env vars should be set.
896 #TARGET_USER='${DEFAULT_TARGET_USER}'
897 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
898 # e.g. for cloud files backend it might look like this (uncomment for use!)
899 #export CLOUDFILES_USERNAME='someuser'
900 #export CLOUDFILES_APIKEY='somekey'
901 #export CLOUDFILES_AUTHURL ='someurl'
902 # the following is an incomplete list (<backend>: comma separated env vars list)
903 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
904 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
905 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
906 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
907 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
908 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
909 # SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
910
911 # base directory to backup
912 SOURCE='${DEFAULT_SOURCE}'
913
914 # a command that runs duplicity e.g.
915 # shape bandwidth use via trickle
916 # "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
917 #DUPL_PRECMD=""
918
919 # override the used python interpreter, defaults to
920 # - parsed result of duplicity's shebang or 'python2'
921 # e.g. "python2" or "/usr/bin/python2.7"
922 #PYTHON="python"
923
924 # exclude folders containing exclusion file (since duplicity 0.5.14)
925 # Uncomment the following two lines to enable this setting.
926 #FILENAME='.duplicity-ignore'
927 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
928
929 # Time frame for old backups to keep, Used for the "purge" command.
930 # see duplicity man page, chapter TIME_FORMATS)
931 #MAX_AGE=1M
932
933 # Number of full backups to keep. Used for the "purgeFull" command.
934 # See duplicity man page, action "remove-all-but-n-full".
935 #MAX_FULL_BACKUPS=1
936
937 # Number of full backups for which incrementals will be kept for.
938 # Used for the "purgeIncr" command.
939 # See duplicity man page, action "remove-all-inc-of-but-n-full".
940 #MAX_FULLS_WITH_INCRS=1
941
942 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3)
943 # forces a full backup if last full backup reaches a specified age, for the
944 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
945 # Uncomment the following two lines to enable this setting.
946 #MAX_FULLBKP_AGE=1M
947 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE "
948
949 # sets duplicity --volsize option (available since v0.4.3.RC7)
950 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
951 # VOLSIZE must be number of MB's to set the volume size to.
952 # Uncomment the following two lines to enable this setting.
953 #VOLSIZE=50
954 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
955
956 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
957 # default is 4, if not set
958 #VERBOSITY=5
959
960 # temporary file space. at least the size of the biggest file in backup
961 # for a successful restoration process. (default is '/tmp', if not set)
962 #TEMP_DIR=/tmp
963
964 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds
965 # unencrypted meta data of the backup, enabling new incrementals without the
966 # need to decrypt backend metadata first. If empty or deleted somehow, the
967 # private key and it's password are needed.
968 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite
969 # big over time so you might want to put it not in the home dir.
970 # default '~/.cache/duplicity/duply_<profile>/'
971 # if set '\${ARCH_DIR}/<profile>'
972 #ARCH_DIR=/some/space/safe/.duply-cache
973
974 # DEPRECATED setting
975 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users
976 # to change the time separator from ':' to another character that will work
977 # on their system. HINT: For Windows SMB shares, use --time-separator='_'.
978 # NOTE: '-' is not valid as it conflicts with date separator.
979 # ATTENTION: only use this with duplicity < 0.5.10, since then default file
980 # naming is compatible and this option is pending depreciation
981 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
982
983 # DEPRECATED setting
984 # activates duplicity --short-filenames option, when uploading to a file
985 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
986 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
987 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file
988 # naming is compatible and this option is pending depreciation
989 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
990
991 # more duplicity command line options can be added in the following way
992 # don't forget to leave a separating space char at the end
993 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here "
994
995 EOF
996
997 # create initial exclude file
998 cat <<EOF >"$EXCLUDE"
999 # although called exclude, this file is actually a globbing file list
1000 # duplicity accepts some globbing patterns, even including ones here
1001 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1002 # - dir/bar/foo
1003 # + dir/bar
1004 # - **
1005 # for more details see duplicity manpage, section File Selection
1006 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1007
1008 EOF
1009
1010 # Hints on first usage
1011 cat <<EOF
1012
1013 Congratulations. You just created the profile '$FTPLCFG'.
1014 The initial config file has been created as
1015 '$CONF'.
1016 You should now adjust this config file to your needs.
1017
1018 $(hint_profile)
1019
1020 EOF
1021 fi
1022
1023 }
1024
1025 # used in usage AND create_config
1026 function hint_profile {
1027 cat <<EOF
1028 IMPORTANT:
1029 Copy the _whole_ profile folder after the first backup to a safe place.
1030 It contains everything (duply related) needed to restore your backups.
1031
1032 Pay attention to (possibly later added) external files such as credentials
1033 or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1034 (e.g. DPBX_ACCESS_TOKEN).
1035 It is good policy to place those in the profile folder if possible at all.
1036 e.g. in case of 'multi://' target the config .json file
1037 Env vars should be added to duply profiles' conf file.
1038
1039 Keep access to these files restricted as they contain information (gpg key,
1040 passphrases etc.) to access and modify your backups.
1041
1042 Finally:
1043 You should attempt a restore from an unrelated host to be sure you really
1044 have everything needed for restoration.
1045
1046 Repeat these steps after _all_ configuration changes. Some configuration
1047 options are crucial for restoration.
1048
1049 EOF
1050 }
1051
1052 function separator {
1053 echo "--- $@ ---"
1054 }
1055
1056 function inform {
1057 echo -e "\nINFO:\n\n$@\n"
1058 }
1059
1060 function warning {
1061 echo -e "\nWARNING:\n\n$@\n"
1062 }
1063
1064 function warning_oldhome {
1065 local old=$1 new=$2
1066 warning " ftplicity changed name to duply since you created your profiles.
1067 Please rename the old folder
1068 '$old'
1069 to
1070 '$new'
1071 and this warning will disappear.
1072 If you decide not to do so profiles will _only_ work from the old location."
1073 }
1074
1075 function error_print {
1076 echo -e "$@" >&2
1077 }
1078
1079 function error {
1080 error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1081 exit -1
1082 }
1083
1084 function error_gpg {
1085 [ -n "$2" ] && local hint="\n $2\n\n "
1086
1087 error "$1
1088
1089 Hint${hint:+s}:
1090 ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1091 Don't forget the used _password_ as you will need it.
1092 When done enter the 8 digit id & the password in the profile conf file.
1093
1094 The key id can be found doing a 'gpg --list-keys'. In the example output
1095 below the key id for the public key would be FFFFFFFF.
1096
1097 pub 1024D/FFFFFFFF 2007-12-17
1098 uid duplicity
1099 sub 2048g/899FE27F 2007-12-17
1100 "
1101 }
1102
1103 function error_gpg_test {
1104 [ -n "$2" ] && local hint="\n $2\n\n "
1105
1106 error "$1
1107
1108 Hint${hint:+s}:
1109 ${hint}This error means that gpg is probably misconfigured or not working
1110 correctly. The error message above should help to solve the problem.
1111 However, if for some reason $ME_NAME should misinterpret the situation you
1112 can define GPG_TEST='disabled' in the conf file to bypass the test.
1113 Please do not forget to report the bug in order to resolve the problem
1114 in future versions of $ME_NAME.
1115 "
1116 }
1117
1118 function error_path {
1119 error "$@
1120 PATH='$PATH'
1121 "
1122 }
1123
1124 function error_to_string {
1125 [ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1126 }
1127
1128 function duplicity_version_get {
1129 # nothing to do, just print
1130 var_isset DUPL_VERSION && return
1131
1132 local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1133 # only run with a user specific python if configured (running by default
1134 # breaks homebrew as they place a shell wrapper for duplicity in path)
1135 PYTHON_BIN="$(python_binary)" &&\
1136 CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
1137 CMD="$CMD --version 2>&1"
1138 DUPL_VERSION_OUT=$(eval "$CMD")
1139 DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1140 #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1141 DUPL_VERSION_VALUE=0
1142 DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1143 if (match(v,/[^\.0-9]+[0-9]*$/)){
1144 rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1145 if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1146 split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1147 print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1148 print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1149 print "DUPL_VERSION_RC=\047"rc"\047"
1150 print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1151 }')
1152 eval "$DUPL_VERSION_AWK"
1153 #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1154
1155 # doublecheck findings and report error
1156 if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1157 inform "duplicity version check failed (please report, this is a bug)
1158 the command
1159 $CMD
1160 resulted in
1161 $DUPL_VERSION_OUT
1162 "
1163 elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1164 error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1165 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1166 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1167 fi
1168 }
1169
1170 function duplicity_version_ge {
1171 [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1172 }
1173
1174 function duplicity_version_lt {
1175 ! duplicity_version_ge "$1"
1176 }
1177
1178 # parse interpreter from duplicity shebang
1179 function duplicity_python_binary_parse {
1180 # cached result
1181 ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
1182
1183 # parse it or warn
1184 local DUPL_BIN=$(which duplicity)
1185 DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1186 if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1187 warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
1188 Will assume it is '$DEFAULT_PYTHON'."
1189 DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
1190 fi
1191 }
1192
1193 function run_script { # run pre/post scripts
1194 local ERR=0
1195 local SCRIPT="$1"
1196 if [ ! -z "$PREVIEW" ] ; then
1197 echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1198 elif [ -r "$SCRIPT" ] ; then
1199 echo -n "Running '$SCRIPT' "
1200 if [ -x "$SCRIPT" ]; then
1201 OUT=$("$SCRIPT" 2>&1)
1202 ERR=$?
1203 else
1204 OUT=$(. "$SCRIPT" 2>&1)
1205 ERR=$?
1206 fi
1207 [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1208 echo -en ${OUT:+"Output: $OUT\n"} ;
1209 else
1210 echo "Skipping n/a script '$SCRIPT'."
1211 fi
1212 return $ERR
1213 }
1214
1215 function run_cmd {
1216 # run or print escaped cmd string
1217 local CMD_ERR=0
1218 if [ -n "$PREVIEW" ]; then
1219 CMD_OUT=$( echo "$@ 2>&1" )
1220 CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1221 elif [ -n "$CMD_DISABLED" ]; then
1222 CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1223 else
1224 echo -n -e "$CMD_MSG"
1225 CMD_OUT=` eval "$@" 2>&1 `
1226 CMD_ERR=$?
1227 if [ "$CMD_ERR" = "0" ]; then
1228 CMD_MSG=" (OK)"
1229 else
1230 CMD_MSG=" (FAILED)"
1231 fi
1232 fi
1233 echo -e "$CMD_MSG"
1234 # reset
1235 unset CMD_DISABLED CMD_MSG
1236 return $CMD_ERR
1237 }
1238
1239 function qw { quotewrap "$@"; }
1240
1241 function quotewrap {
1242 local param="$@"
1243 # quote strings having non word chars (e.g. spaces)
1244 if echo "$param" | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1245 echo "$param" | awk '{\
1246 gsub(/[\047]/,"\047\\\047\047",$0);\
1247 gsub(/[\042]/,"\047\\\042\047",$0);\
1248 print "\047"$0"\047"}'
1249 return
1250 fi
1251 echo $param
1252 }
1253
1254 function duplicity_params_global {
1255 # already done? return
1256 var_isset 'DUPL_PARAMS_GLOBAL' && return
1257 local DUPL_ARG_ENC
1258
1259 # use key only if set in config, else leave it to symmetric encryption
1260 if gpg_disabled; then
1261 local DUPL_PARAM_ENC='--no-encryption'
1262 else
1263 local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1264 gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1265 # interpret password settings
1266 var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1267 var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1268 fi
1269
1270 local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1271
1272 # set name for dupl archive folder, since 0.6.0
1273 if duplicity_version_ge 601; then
1274 local DUPL_ARCHDIR=''
1275 if var_isset 'ARCH_DIR'; then
1276 DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1277 # reuse erronously duply_ prefixed folders from bug #117
1278 if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1279 DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1280 else
1281 DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1282 fi
1283 else
1284 DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1285 fi
1286 fi
1287
1288 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1289 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1290 ${GPG_OPTS}"
1291
1292 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1293 ${DUPL_ARG_ENC}"
1294 }
1295
1296 # function to filter the DUPL_PARAMS var from user conf
1297 function duplicity_params_conf {
1298 # reuse cmd var from main loop
1299 ## in/exclude parameters are currently not supported on restores
1300 if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1301 # filter exclude params from fetch/restore/status
1302 eval "stripXcludes $DUPL_PARAMS"
1303 return
1304 fi
1305
1306 # nothing done, print unchanged
1307 echo "$DUPL_PARAMS"
1308 }
1309
1310 # strip in/exclude parameters from param string
1311 function stripXcludes {
1312 local STRIPNEXT OUT;
1313 for p in "$@"; do
1314 if [ -n "$STRIPNEXT" ]; then
1315 unset STRIPNEXT
1316 # strip the value of previous parameter
1317 continue
1318 elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1319 # strips e.g. --include /foo/bar
1320 STRIPNEXT="yes"
1321 continue
1322 elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1323 # strips e.g. --include=/foo/bar
1324 continue
1325 fi
1326
1327 OUT="$OUT $(qw "$p")"
1328 done
1329 echo "$OUT"
1330 }
1331
1332 function duplify { # the actual wrapper function
1333 local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1334
1335 # put command (with params) first in duplicity parameters
1336 for param in "$@" ; do
1337 # split cmd from params (everything before splitchar --)
1338 if [ "$param" == "--" ] ; then
1339 PARAMSNOW=1
1340 else
1341 # wrap in quotes to protect from spaces
1342 [ ! $PARAMSNOW ] && \
1343 DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1344 || \
1345 DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1346 fi
1347 done
1348
1349 # init global duplicity parameters same for all tasks
1350 duplicity_params_global
1351
1352 local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
1353 # run in cmd line preview mode if requested
1354 var_isset 'PREVIEW' && RUN=echo
1355 # try to resolve duplicity path for usage with python interpreter
1356 DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
1357 # only run with a user specific python if configured (running by default
1358 # breaks homebrew as they place a shell wrapper for duplicity in path)
1359 PYTHON_BIN="$(python_binary)" &&\
1360 BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
1361
1362 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1363 ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1364 $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1365
1366 local ERR=$?
1367 return $ERR
1368 }
1369
1370 function secureconf { # secure the configuration dir
1371 #PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1372 local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1373 if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1374 chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1375 warning "The profile's folder
1376 '$CONFDIR'
1377 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1378 fi
1379 }
1380
1381 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1382 function date_fix {
1383 local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1384 local date
1385 #[ "$1" == "%N" ] && return #test the no nsec test below
1386 # gnu date with -d @epoch
1387 date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1388 echo $date && return
1389 # date bsd,osx with -r epoch
1390 date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1391 echo $date && return
1392 # date busybox with -d epoch -D %s
1393 date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1394 echo $date && return
1395 ## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1396 # python fallback
1397 date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1398 echo $date && return
1399 # awk fallback
1400 date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1401 echo $date && return
1402 # perl fallback
1403 date=$(perl -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1404 echo $date && return
1405 # error
1406 echo "ERROR"
1407 return 1
1408 }
1409
1410 function nsecs {
1411 local NSECS
1412 # test if date supports nanosecond output
1413 if ! var_isset NSECS_DISABLED; then
1414 NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1415 [ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1416 fi
1417
1418 # add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1419 if [ "$NSECS_DISABLED" == "1" ]; then
1420 date_fix %s000000000
1421 else
1422 date_fix %s%N
1423 fi
1424 }
1425
1426 function nsecs_to_sec {
1427 echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1428 }
1429
1430 function datefull_from_nsecs {
1431 date_from_nsecs $1 '%F %T'
1432 }
1433
1434 function date_from_nsecs {
1435 local FORMAT=${2:-%T}
1436 local TIME=$(nsecs_to_sec $1)
1437 local SECS=${TIME%.*}
1438 local DATE=$(date_fix "%T" ${SECS:-0})
1439 echo $DATE.${TIME#*.}
1440 }
1441
1442 function var_isset {
1443 if [ -z "$1" ]; then
1444 echo "ERROR: function var_isset needs a string as parameter"
1445 elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1446 return 0
1447 fi
1448 return 1
1449 }
1450
1451 function is_condition {
1452 local CMD=$(tolower "$1")
1453 [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1454 }
1455
1456 function is_groupMarker {
1457 local CMD=$(tolower "$1")
1458 [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1459 }
1460
1461 function is_command {
1462 local CMD=$(tolower "$1")
1463 ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1464 }
1465
1466 function url_encode {
1467 # utilize python, silently do nothing on error - because no python no duplicity
1468 OUT=$("$(python_binary)" -c "
1469 try: import urllib.request as urllib
1470 except ImportError: import urllib
1471 print(urllib.${2}quote('$1'));
1472 " 2>/dev/null ); ERR=$?
1473 [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1474 }
1475
1476 function url_decode {
1477 # reuse function above with a simple string param hack
1478 url_encode "$1" "un"
1479 }
1480
1481 function toupper {
1482 echo "$@"|awk '$0=toupper($0)'
1483 }
1484
1485 function tolower {
1486 echo "$@"|awk '$0=tolower($0)'
1487 }
1488
1489 function isnumber {
1490 case "$*" in
1491 ''|*[!0-9]*) return 1;;
1492 *) return 0;;
1493 esac
1494 }
1495
1496 #function tmp_space {
1497 #
1498 # if ! isnumber $VOLSIZE; then
1499 # inform "failed to determine free space (please report, this is a bug)"
1500 # return
1501 # fi
1502 #
1503 # get free temp space
1504 # TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1505 # # check for free space or FAIL
1506 # if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1507 # error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1508 #than one duplicity volume (${VOLSIZE}MB).
1509 #
1510 # Hint: Free space or change TEMP_DIR setting."
1511 #fi
1512 #
1513 #}
1514
1515 function gpg_disabled {
1516 echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1517 }
1518
1519 # usage: join SEPARATOR "entry1" "entry2"
1520 function join {
1521 local SEP="$1" ENTRY OUT; shift;
1522 for ENTRY in "$@"; do
1523 ENTRY=${ENTRY//$SEP/\\$SEP}
1524 [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1525 done
1526 echo $OUT
1527 }
1528
1529 function gpg_testing {
1530 [ "$GPG_TEST" != "disabled" ]
1531 }
1532
1533 function gpg_signing {
1534 echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1535 }
1536
1537 function gpg_keytype {
1538 echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1539 }
1540
1541 # parameter key id, key_type
1542 function gpg_keyfile {
1543 local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1544 local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1545 echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1546 }
1547
1548 # parameter key id
1549 function gpg_import {
1550 local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1551 [ "$GPG_IMPORT" = "disabled" ] && {
1552 echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1553 return
1554 }
1555
1556 # create a list of legacy key file names and current naming scheme
1557 # we always import pub and sec if they are avail in conf folder
1558 local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1559 $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1560
1561 # Try autoimport from existing old gpgkey files
1562 # and new gpgkey.XXX.asc files (since v1.4.2)
1563 # and even newer gpgkey.XXX.[pub|sec].asc
1564 for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1565 FILE=${KEYFILES[$i]}
1566 if [ -f "$FILE" ]; then
1567 FOUND=1
1568
1569 CMD_MSG="Import keyfile '$FILE' to keyring"
1570 run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1571 if [ "$?" != "0" ]; then
1572 warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1573 ERR=1
1574 # continue with next
1575 continue
1576 fi
1577 fi
1578 done
1579
1580 if [ "$FOUND" -eq 0 ]; then
1581 echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1582 return 1
1583 fi
1584
1585 # try to set trust automagically
1586 CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1587 run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1588 if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then
1589 # success on all levels, we're done
1590 return $ERR
1591 fi
1592
1593 # failover: user has to set trust manually
1594 echo -e "For $ME_NAME to work you have to set the trust level
1595 with the command \"trust\" to \"ultimate\" (5) now.
1596 Exit the edit mode of gpg with \"quit\"."
1597 CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1598 run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1599
1600 return $ERR
1601 }
1602
1603 # see 'How to specify a user ID' on gpg manpage
1604 function gpg_fingerprint {
1605 gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1606 awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1607 }
1608
1609 function gpg_export_if_needed {
1610 [ "$GPG_EXPORT" = 'disabled' ] && { \
1611 echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1612 return
1613 }
1614
1615 local SUCCESS FILE KEY_TYPE
1616 local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1617 for KEY_ID in "$@"; do
1618 # check if already exported, do it if not
1619 for KEY_TYPE in PUB SEC; do
1620 FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1621 if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1622
1623 # exporting
1624 CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1625 # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1626 # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1627 run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1628 CMD_ERR=$?
1629
1630 if [ "$CMD_ERR" = "0" ]; then
1631 CMD_MSG="Write file '"$(basename "$FILE")"'"
1632 run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1633 fi
1634
1635 if [ "$CMD_ERR" != "0" ]; then
1636 warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1637 else
1638 SUCCESS=1
1639 fi
1640
1641 # cleanup
1642 rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1643 fi
1644 done
1645 done
1646
1647 [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1648 You should backup your changed profile folder now and store it in a safe place."
1649 }
1650
1651 # replace all non-alnum chars with underscore (for file operations)
1652 function gpg_key_legalize {
1653 echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1654 }
1655
1656 function gpg_key_cache {
1657 local RES
1658 local MODE=$1
1659 shift
1660 local PREFIX="GPG_KEY"
1661 local SUFFIX=$(gpg_key_legalize "$@")
1662 local KEYID="$*"
1663 local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1664 if [ "$MODE" = "RESET" ]; then
1665 eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1666 return 255
1667 elif ! var_isset "$CACHE"; then
1668 if [ "$MODE" = "PUB" ]; then
1669 RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1670 elif [ "$MODE" = "SEC" ]; then
1671 RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1672 else
1673 return 255
1674 fi
1675 eval $CACHE=$RES
1676 fi
1677 eval return \$$CACHE
1678 }
1679
1680 function gpg_pub_avail {
1681 gpg_key_cache PUB "$@"
1682 }
1683
1684 function gpg_sec_avail {
1685 gpg_key_cache SEC "$@"
1686 }
1687
1688 function gpg_key_format {
1689 echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1690 }
1691
1692 # splits a comma separated line into lines, respects escaped commas
1693 function gpg_split_keyset {
1694 local LIST
1695 LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1696 echo -e "$LIST"
1697 }
1698
1699 function gpg_prefix_keyset {
1700 local PREFIX="$1" OUT=""
1701 shift
1702 for KEY_ID in "$@"; do
1703 OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1704 done
1705 echo $OUT
1706 }
1707
1708 # grep a variable from conf text file (currently not used)
1709 function gpg_passwd {
1710 [ -r "$CONF" ] && \
1711 awk '/^[ \t]*GPG_PW[ \t=]/{\
1712 sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1713 gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1714 print $0; exit}' "$CONF"
1715 }
1716
1717 # return success if at least one secret key is available
1718 function gpg_key_decryptable {
1719 local KEY_ID
1720 for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1721 gpg_sec_avail "$KEY_ID" && return 0
1722 done
1723 return 1
1724 }
1725
1726 function gpg_symmetric {
1727 [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1728 }
1729
1730 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1731 function gpg_param_passwd {
1732 var_isset GPG_USEAGENT && exit 1
1733
1734 if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1735 echo "--passphrase-fd 0 --batch"
1736 fi
1737 }
1738
1739 # select the earliest defined and create an "echo <value> |" string
1740 function gpg_pass_pipein {
1741 var_isset GPG_USEAGENT && exit 1
1742
1743 for var in "$@"
1744 do
1745 if var_isset "$var"; then
1746 echo "echo $(qw $(eval echo \$$var)) |"
1747 return 0
1748 fi
1749 done
1750
1751 return 1
1752 }
1753
1754 # checks if gpg-agent is available, returns error code
1755 # 0 on success
1756 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1757 # 2 if GPG_AGENT_INFO is stale
1758 # 3 cannot connect to gpg-agent
1759 function gpg_agent_avail {
1760 # GPG_AGENT_INFO is deprecated in gpg2.1,
1761 # first try to connect to a possibly running agent here
1762 local ERR=3
1763 gpg-agent > /dev/null 2>&1 && return 0
1764
1765 # detect stale pid in legacy GPG_AGENT_INFO env var
1766 if var_isset GPG_AGENT_INFO; then
1767 # check if a pid matching process is running at all
1768 local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1769 if isnumber "$GPG_AGENT_PID"; then
1770 ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1771 fi
1772 fi
1773
1774 return $ERR
1775 }
1776
1777 function gpg_custom_binary {
1778 var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1779 echo "--gpg-binary $(qw "$GPG")"
1780 }
1781
1782 function gpg_binary {
1783 local BIN
1784 var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1785 echo "$BIN"
1786 }
1787
1788 function gpg_avail {
1789 lookup "$(gpg_binary)"
1790 }
1791
1792 # enforce the use of our selected gpg binary
1793 function gpg {
1794 command "$(gpg_binary)" "$@"
1795 }
1796 export -f gpg
1797
1798 # start of script #######################################################################
1799
1800 # confidentiality first, all we create is only readable by us
1801 umask 077
1802
1803 # check if ftplicity is there & executable
1804 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1805
1806 if [ ${#@} -eq 1 ]; then
1807 cmd="${1}"
1808 else
1809 FTPLCFG="${1}" ; cmd="${2}"
1810 fi
1811
1812 # deal with command before profile validation calls
1813 # show requested version
1814 # OR requested usage info
1815 # OR create a profile
1816 # OR fall through
1817 ##if [ ${#@} -le 2 ]; then
1818 case "$cmd" in
1819 changelog)
1820 changelog
1821 exit 0
1822 ;;
1823 create)
1824 set_config
1825 if [ -d "$CONFDIR" ]; then
1826 error "The profile '$FTPLCFG' already exists in
1827 '$CONFDIR'.
1828
1829 Hint:
1830 If you _really_ want to create a new profile by this name you will
1831 have to manually delete the existing profile folder first."
1832 exit 1
1833 else
1834 create_config
1835 exit 0
1836 fi
1837 ;;
1838 txt2man)
1839 set_config
1840 usage_txt2man
1841 exit 0
1842 ;;
1843 usage|-help|--help|-h|-H)
1844 set_config
1845 usage_info
1846 exit 0
1847 ;;
1848 version|-version|--version|-v|-V)
1849 # profile can override GPG/PYTHON, so import it if it was given
1850 var_isset FTPLCFG && {
1851 set_config
1852 [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1853 }
1854 version_info_using
1855 exit 0
1856 ;;
1857 # fallthrough.. we got a command that needs an existing profile
1858 *)
1859 # if we reach here, user either forgot profile or chose wrong profileless command
1860 if [ ${#@} -le 1 ]; then
1861 error "\
1862 Missing or wrong parameters.
1863 Only the commands
1864 changelog, create, usage, txt2man, version
1865 can be called without selecting an existing profile first.
1866 Your command was '$cmd'.
1867
1868 Hint: Run '$ME usage' to get help."
1869 fi
1870 esac
1871
1872
1873 # Hello world
1874 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1875
1876 # check system environment
1877
1878 # is duplicity avail
1879 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1880 # init, exec duplicity version check info
1881 duplicity_python_binary_parse
1882 duplicity_version_get
1883
1884 # check for certain important helper programs
1885 for f in awk grep "$(python_binary)"; do
1886 lookup "$f" || \
1887 error_path "$f missing. installed und available in path?"
1888 done
1889
1890 ### read configuration
1891 set_config
1892 # check validity
1893 if [ ! -d "$CONFDIR" ]; then
1894 error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1895 '$CONFDIR'.
1896
1897 Hints:
1898 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1899 Use '$ME <name> create' to create a new profile.
1900 Use '$ME usage' to get usage help."
1901 elif [ ! -x "$CONFDIR" ]; then
1902 error "\
1903 Profile folder in '$CONFDIR' cannot be accessed.
1904
1905 Hint:
1906 Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1907 elif [ ! -f "$CONF" ] ; then
1908 error "'$CONF' not found."
1909 elif [ ! -r "$CONF" ] ; then
1910 error "'$CONF' not readable."
1911 else
1912 . "$CONF"
1913 #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1914 TEMP_DIR=${TEMP_DIR:-'/tmp'}
1915 # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1916 if var_isset 'TARGET_PW'; then
1917 TARGET_PASS="${TARGET_PW}"
1918 fi
1919 fi
1920 echo "Using profile '$CONFDIR'."
1921
1922 # secure config dir, if needed w/ warning
1923 secureconf
1924
1925 # split TARGET in handy variables
1926 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1927 target=$0; match(target,/^([^\/:]+):\/\//); \
1928 prot=substr(target,RSTART,RLENGTH);\
1929 rest=substr(target,RSTART+RLENGTH); \
1930 if (credsavail=match(rest,/^[^@]*@/)){\
1931 creds=substr(rest,RSTART,RLENGTH-1);\
1932 credcount=split(creds,cred,":");\
1933 rest=substr(rest,RLENGTH+1);\
1934 # split creds with regexp\
1935 match(creds,/^([^:]+)/);\
1936 user=substr(creds,RSTART,RLENGTH);\
1937 pass=substr(creds,RSTART+1+RLENGTH);\
1938 };\
1939 # filter quotes or escape them\
1940 gsub(/[\047\042]/,"",prot);\
1941 gsub(/[\047\042]/,"",rest);\
1942 gsub(/[\047]/,"\047\\\047\047",creds);\
1943 print "TARGET_URL_PROT=\047"prot"\047\n"\
1944 "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1945 "TARGET_URL_CREDS=\047"creds"\047\n";\
1946 if(user){\
1947 gsub(/[\047]/,"\047\\\047\047",user);\
1948 print "TARGET_URL_USER=\047"user"\047\n"}\
1949 if(pass){\
1950 gsub(/[\047]/,"\047\\\047\047",pass);\
1951 print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1952 }')
1953 eval "${TARGET_SPLIT_URL}"
1954
1955 # fetch commmand from parameters ########################################################
1956 # Hint: cmds is also used to check if authentification info sufficient in the next step
1957 cmds="$2"; shift 2
1958
1959 # complain if command(s) missing
1960 [ -z $cmds ] && error " No command given.
1961
1962 Hint:
1963 Use '$ME usage' to get usage help."
1964
1965 # process params
1966 for param in "$@"; do
1967 #echo !$param!
1968 case "$param" in
1969 # enable ftplicity preview mode
1970 '--preview')
1971 PREVIEW=1
1972 ;;
1973 # interpret duplicity disable encr switch
1974 '--disable-encryption')
1975 GPG_KEY='disabled'
1976 ;;
1977 *)
1978 if [ `echo "$param" | grep -e "^-"` ] || \
1979 [ `echo "$last_param" | grep -e "^-"` ] ; then
1980 # forward parameter[/option pairs] to duplicity
1981 dupl_opts["${#dupl_opts[@]}"]=${param}
1982 else
1983 # anything else must be a parameter (e.g. for fetch, ...)
1984 ftpl_pars["${#ftpl_pars[@]}"]=${param}
1985 fi
1986 last_param=${param}
1987 ;;
1988 esac
1989 done
1990
1991 # plausibility check config - VARS & KEY ################################################
1992 # check if src, trg, trg pw
1993 # auth info sufficient
1994 # gpg key, gpg pwd (might be empty) set in config
1995 # OR key in local gpg db
1996 # OR key can be imported from keyfile
1997 # OR fail
1998 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
1999 error " Source Path (setting SOURCE) not set or still default value in conf file
2000 '$CONF'."
2001
2002 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2003 error " Backup Target (setting TARGET) not set or still default value in conf file
2004 '$CONF'."
2005
2006 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2007 [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2008 error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}')
2009 are configured with different values. There can be only one.
2010
2011 Hint: Remove conflicting setting."
2012
2013 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2014 [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2015 error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}')
2016 are configured with different values. There can be only one.
2017
2018 Hint: Remove conflicting setting."
2019 fi
2020
2021 # GPG config plausibility check1 (disabled check) #############################
2022 if gpg_disabled; then
2023 : # encryption disabled, all is well
2024 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2025 warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file
2026 '$CONF'.
2027 Will disable encryption for duplicity now.
2028
2029 Hint:
2030 If you really want to use _no_ encryption you can disable this warning by
2031 setting GPG_KEY='disabled' in conf file."
2032 GPG_KEY='disabled'
2033 fi
2034
2035 # GPG availability check (now we know if gpg is really needed)#################
2036 if ! gpg_disabled; then
2037 gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2038 fi
2039
2040 # Output versions info ########################################################
2041 using_info
2042
2043 # GPG create key settings, config check2 (needs gpg) ##########################
2044 if gpg_disabled; then
2045 : # the following tests are not necessary
2046 else
2047
2048 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2049 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2050
2051 # enc key still default?
2052 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then
2053 error_gpg "Encryption Key GPG_KEY still default in conf file
2054 '$CONF'."
2055 fi
2056
2057 # create array of gpg encr keys, for further processing
2058 OIFS="$IFS" IFS=$'\n'
2059 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2060 IFS="$OIFS"
2061
2062 # pw set?
2063 # symmetric needs one, always
2064 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2065 ; then
2066 error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption)
2067 is empty/not set or still default value in conf file
2068 '$CONF'."
2069 fi
2070 # this is a technicality, we can only pump one pass via pipe into gpg
2071 # but symmetric already always needs one for encryption
2072 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2073 [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2074 error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2075 '$CONF'.
2076 This is unfortunately impossible. For details see duplicity manpage,
2077 section 'A Note On Symmetric Encryption And Signing'.
2078
2079 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2080 Tip2: Use gpg-agent."
2081 fi
2082
2083 # test - GPG KEY AVAILABILITY ##################################################
2084
2085 # check gpg public keys availability, try import if needed
2086 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2087 KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2088 # test availability, try to import, retest
2089 if ! gpg_pub_avail "${KEY_ID}"; then
2090 echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2091 gpg_import "${KEY_ID}" PUB
2092 gpg_key_cache RESET "${KEY_ID}"
2093 gpg_pub_avail "${KEY_ID}" || { \
2094 gpg_testing && error_gpg \
2095 "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2096 "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2097 as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2098 If not you can put it there and $ME_NAME will autoimport it on the next run.
2099 Alternatively import it manually as the user you plan to run $ME_NAME with."
2100 }
2101 else
2102 echo "Public key '${KEY_ID}' found in keychain."
2103 fi
2104 done
2105
2106 # check gpg encr secret encryption keys availability and fail
2107 # if none is available after a round of importing trials
2108 gpg_key_decryptable || \
2109 {
2110 echo "Missing secret keys for decryption in keychain."
2111 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2112 KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2113 # test availability, try to import, retest
2114 if ! gpg_sec_avail "${KEY_ID}"; then
2115 echo "Try to import secret key '${KEY_ID}' from profile."
2116 gpg_import "${KEY_ID}" SEC
2117 gpg_key_cache RESET "${KEY_ID}"
2118 fi
2119 done
2120 gpg_key_decryptable || \
2121 {
2122 gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2123 has a secret key in the keychain. Decryption will be impossible!"
2124 }
2125 }
2126
2127 # gpg secret sign key availability
2128 # if none set, autoset first encryption key as sign key
2129 if ! gpg_signing; then
2130 echo "Signing disabled per configuration."
2131 # try first key, if one set
2132 elif ! var_isset 'GPG_KEY_SIGN'; then
2133 KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2134 if [ -z "${KEY_ID}" ]; then
2135 echo "Signing disabled. No GPG_KEY entries in config."
2136 GPG_KEY_SIGN='disabled'
2137 else
2138 # use avail OR try import OR fail
2139 if gpg_sec_avail "${KEY_ID}"; then
2140 GPG_KEY_SIGN="${KEY_ID}"
2141 else
2142 echo "Signing secret key '${KEY_ID}' not found."
2143 gpg_import "${KEY_ID}" SEC
2144 gpg_key_cache RESET "${KEY_ID}"
2145 if gpg_sec_avail "${KEY_ID}"; then
2146 GPG_KEY_SIGN="${KEY_ID}"
2147 fi
2148 fi
2149
2150 # interpret sign key setting
2151 if var_isset 'GPG_KEY_SIGN'; then
2152 echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2153 else
2154 echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2155 GPG_KEY_SIGN='disabled'
2156 fi
2157 fi
2158 else
2159 KEY_ID="${GPG_KEY_SIGN}"
2160 if ! gpg_sec_avail "${KEY_ID}"; then
2161 inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2162 gpg_import "${KEY_ID}" SEC
2163 gpg_key_cache RESET "${KEY_ID}"
2164 gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2165 fi
2166 fi
2167
2168 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2169 # key enc can deal without, but might profit from gpg-agent
2170 # if GPG_PW is not set alltogether
2171 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2172 if ! gpg_symmetric && \
2173 ( ! var_isset GPG_PW || \
2174 ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2175
2176 if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2177 warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2178 elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2179 warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2180 elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2181 warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2182 else
2183 echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2184 GPG_USEAGENT="--use-agent"
2185 fi
2186 fi
2187
2188 # end GPG config plausibility check2
2189 fi
2190
2191 # config plausibility check - SPACE ###########################################
2192
2193 # is tmp is a folder and writable
2194 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2195 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2196 if [ "$?" != "0" ]; then
2197 error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2198 fi
2199
2200
2201 # get volsize, default duplicity volume size is 25MB since v0.5.07
2202 VOLSIZE=${VOLSIZE:-25}
2203 # double if asynch is on
2204 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2205
2206 # TODO: check for enough (async= upload space and WARN only
2207 # use function tmp_space
2208 #echo TODO: reimplent tmp space check
2209
2210
2211 # test - GPG SANITY #####################################################################
2212 # if encryption is disabled, skip this whole section
2213 if gpg_disabled; then
2214 echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2215 elif ! gpg_testing; then
2216 echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2217 else
2218
2219 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2220 function cleanup_gpgtest {
2221 echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2222 rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2223 }
2224
2225 # signing enabled?
2226 if gpg_signing; then
2227 CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2228 CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2229 fi
2230
2231 # using keys
2232 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2233
2234 for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2235 CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2236 done
2237 # check encrypting
2238 CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2239 run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2240 CMD_ERR=$?
2241
2242 if [ "$CMD_ERR" != "0" ]; then
2243 KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2244 [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2245 'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2246 can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2247 error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2248 fi
2249
2250 # check decrypting
2251 CMD_MSG="Test - Decrypt"
2252 gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2253 run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2254 CMD_ERR=$?
2255
2256 if [ "$CMD_ERR" != "0" ]; then
2257 error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2258 fi
2259
2260 # symmetric only
2261 else
2262 # check encrypting
2263 CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2264 run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2265 CMD_ERR=$?
2266 if [ "$CMD_ERR" != "0" ]; then
2267 error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2268 fi
2269
2270 # check decrypting
2271 CMD_MSG="Test - Decryption with passphrase"
2272 run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2273 CMD_ERR=$?
2274 if [ "$CMD_ERR" != "0" ]; then
2275 error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2276 fi
2277 fi
2278
2279 # compare original w/ decryptginal
2280 CMD_MSG="Test - Compare"
2281 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2282 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2283 CMD_ERR=$?
2284 if [ "$CMD_ERR" = "0" ]; then
2285 cleanup_gpgtest
2286 else
2287 error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2288 fi
2289
2290 fi # end disabled
2291
2292 ## an empty line
2293 #echo
2294
2295 # Exclude file is needed, create it if necessary
2296 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2297
2298 # export only used keys, if bkp not already exists ######################################
2299 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2300
2301
2302 # command execution #####################################################################
2303
2304 # urldecode url vars into plain text
2305 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2306 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2307
2308 # defined TARGET_USER&PASS vars replace their URL pendants
2309 # (double defs already dealt with)
2310 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2311 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2312
2313 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2314
2315 # issue some warnings
2316 case "$TARGET_URL_PROT_lowercase" in
2317 'cf+http')
2318 # info on missing AUTH_URL
2319 if ! var_isset 'CLOUDFILES_AUTHURL'; then
2320 inform "No CLOUDFILES_AUTHURL exported (in conf).
2321 Will use default which is probably rackspace."
2322 fi
2323 ;;
2324 'swift')
2325 # info on possibly missing AUTH_URL
2326 var_isset 'SWIFT_AUTHURL' ||\
2327 warning "\
2328 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2329 ;;
2330 'rsync')
2331 # everything in url (this backend does not support pass in env var)
2332 # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2333 # print warning older version is detected
2334 duplicity_version_lt 610 &&
2335 warning "\
2336 Duplicity version '$DUPL_VERSION' does not support providing the password as
2337 env var for rsync backend. For security reasons you should consider to
2338 update to a version greater than '0.6.10' of duplicity."
2339 ;;
2340 esac
2341
2342
2343 # for all protocols we put username in url and pass into env var
2344 # for sec�rity reasons, we url_encode username to protect special chars
2345 # first sortout backends with special ways to handle password
2346 case "$TARGET_URL_PROT_lowercase" in
2347 'imap'|'imaps')
2348 var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2349 ;;
2350 *)
2351 # rest uses FTP_PASS var
2352 var_isset 'TARGET_URL_PASS' && \
2353 BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2354 ;;
2355 esac
2356 # insert url encoded username into target url if needed
2357 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2358 BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2359 else
2360 BACKEND_URL="$TARGET"
2361 fi
2362
2363
2364 # protect eval from special chars in url (e.g. open ')' in password,
2365 # spaces in path, quotes) happens above in duplify() via quotewrap()
2366 SOURCE="$SOURCE"
2367 BACKEND_URL="$BACKEND_URL"
2368 EXCLUDE="$EXCLUDE"
2369 # since 0.7.03 --exclude-globbing-filelist is deprecated
2370 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist"
2371
2372 # translate backup to batch command
2373 cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
2374
2375 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2376 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2377 gsub(/\+/,\"_and_\",cmds);\
2378 gsub(/\-/,\"_or_\",cmds);\
2379 gsub(/\[/,\"_groupIn_\",cmds);\
2380 gsub(/\]/,\"_groupOut_\",cmds);\
2381 print cmds}")
2382 # convert cmds to array, lowercase for safety
2383 declare -a CMDS
2384 CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
2385
2386 unset FTPL_ERR
2387
2388 # run cmds
2389 for cmd in ${CMDS[*]};
2390 do
2391
2392 ## init
2393 # raise index in cmd array for pre/post param
2394 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2395
2396 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2397
2398 # get next cmd,cnd vars
2399 nextno=$(( $CMD_NO ))
2400 while ! var_isset 'CMD_NEXT'
2401 do
2402 nextno=$(($nextno+1))
2403 if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2404 CMD_VALUE=${CMDS[$nextno]}
2405 is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2406 is_groupMarker "$CMD_VALUE" && continue
2407 CMD_NEXT="$CMD_VALUE"
2408 else
2409 CMD_NEXT='END'
2410 fi
2411 done
2412
2413 # get prev cnd, cnds are skipped pseudocmds
2414 prevno=$(( $CMD_NO ));
2415 while ! var_isset 'CND_PREV'
2416 do
2417 prevno=$(($prevno-1))
2418 if [ "$prevno" -ge 0 ]; then
2419 CMD_VALUE=${CMDS[$prevno]}
2420 is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2421 is_command "$CMD_VALUE" && break
2422 else
2423 break
2424 fi
2425 done
2426
2427 # get prev cmd command minus skipped commands, only executed
2428 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2429 while ! var_isset 'CMD_PREV'
2430 do
2431 prevno=$(($prevno-1))
2432 if [ "$prevno" -ge 0 ]; then
2433 CMD_VALUE=${CMDS[$prevno]}
2434 is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2435 is_groupMarker "$CMD_VALUE" && continue
2436 CMD_PREV="$CMD_VALUE"
2437 else
2438 CMD_PREV='START'
2439 fi
2440 done
2441
2442 function get_cmd_skip_count {
2443 # find closing bracket, get group skip count
2444 local nextno=$CMD_NO
2445 local GRP_OPEN=0
2446 local GRP_SKIP=0
2447 local CMD_VALUE
2448 while [ "$nextno" -lt "${#CMDS[@]}" ]
2449 do
2450 nextno=$(($nextno+1))
2451 CMD_VALUE=${CMDS[$nextno]}
2452 GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2453 if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2454 break;
2455 elif [ "$CMD_VALUE" == 'groupin' ]; then
2456 GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2457 elif [ "$CMD_VALUE" == 'groupout' ]; then
2458 GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2459 if [ "$GRP_OPEN" -lt 1 ]; then
2460 break;
2461 fi
2462 fi
2463 done
2464
2465 echo $GRP_SKIP;
2466 }
2467
2468 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2469 unset SKIP_NOW
2470 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2471 # skip cnd/grp cmds silently
2472 is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2473 CMD_SKIP=$(($CMD_SKIP - 1))
2474 SKIP_NOW="yes"
2475 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2476 CMD_SKIP=$(get_cmd_skip_count)
2477 # incl. this "cmd"
2478 CMD_SKIP=$(( $CMD_SKIP + 1 ))
2479 unset CMD_SKIPPED
2480 SKIP_NOW="yes"
2481 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2482 CMD_SKIP=$(get_cmd_skip_count)
2483 # incl. this "cmd"
2484 CMD_SKIP=$(( $CMD_SKIP + 1 ))
2485 unset CMD_SKIPPED
2486 SKIP_NOW="yes"
2487 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2488 unset 'CMD_SKIP';
2489 SKIP_NOW="yes"
2490 fi
2491
2492 # let's do the skip now
2493 if [ -n "$SKIP_NOW" ]; then
2494 # sum up how many commands we actually skipped for the prev var routines
2495 CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2496 continue
2497 fi
2498
2499 # save start time
2500 RUN_START=$(nsecs)
2501
2502 # export some useful env vars for external scripts/programs to use
2503 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2504 TARGET_URL_USER TARGET_URL_PASS \
2505 GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2506 GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2507 RUN_START
2508
2509 # user info
2510 echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
2511
2512 case "$(tolower $cmd)" in
2513 'pre'|'post')
2514 if [ "$cmd" == 'pre' ]; then
2515 script=$PRE
2516 else
2517 script=$POST
2518 fi
2519 # script execution in a subshell, protect us from failures/var overwrites
2520 ( run_script "$script" )
2521 ;;
2522 'bkp')
2523 duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2524 "$SOURCE" "$BACKEND_URL"
2525 ;;
2526 'incr')
2527 duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2528 "$SOURCE" "$BACKEND_URL"
2529 ;;
2530 'full')
2531 duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2532 "$SOURCE" "$BACKEND_URL"
2533 ;;
2534 'verify')
2535 TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2536 duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2537 "$BACKEND_URL" "$SOURCE"
2538 ;;
2539 'verifypath')
2540 TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2541 IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
2542 ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2543
2544 Hint:
2545 Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2546
2547 duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2548 --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2549 ;;
2550 'list')
2551 # time param exists since 0.5.10+
2552 TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2553 duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2554 ;;
2555 'cleanup')
2556 duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2557 ;;
2558 'purge')
2559 MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2560 [ -z "$MAX_AGE" ] && error " Missing parameter <max_age>. Can be set in profile or as command line parameter."
2561
2562 duplify remove-older-than "${MAX_AGE}" \
2563 -- "${dupl_opts[@]}" "$BACKEND_URL"
2564 ;;
2565 'purgefull')
2566 MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2567 [ -z "$MAX_FULL_BACKUPS" ] && error " Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2568
2569 duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2570 -- "${dupl_opts[@]}" "$BACKEND_URL"
2571 ;;
2572 'purgeincr')
2573 MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2574 [ -z "$MAX_FULLS_WITH_INCRS" ] && error " Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2575
2576 duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2577 -- "${dupl_opts[@]}" "$BACKEND_URL"
2578 ;;
2579 'restore')
2580 OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2581 [ -z "$OUT_PATH" ] && error " Missing parameter target_path for restore.
2582
2583 Hint:
2584 Syntax is -> $ME <profile> restore <target_path> [<age>]"
2585
2586 duplify -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2587 ;;
2588 'fetch')
2589 IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
2590 TIME="${ftpl_pars[2]:-now}";
2591 ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <src_path> or <target_path> for fetch.
2592
2593 Hint:
2594 Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2595
2596 # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2597 duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2598 --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2599 ;;
2600 'status')
2601 duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2602 ;;
2603 *)
2604 error " Unknown command '$cmd'.
2605
2606 Hint:
2607 Use '$ME usage' to get usage help."
2608 ;;
2609 esac
2610
2611 CMD_ERR=$?
2612 RUN_END=$(nsecs)
2613 RUNTIME=$(( $RUN_END - $RUN_START ))
2614
2615 # print message on error; set error code
2616 if [ "$CMD_ERR" -ne 0 ]; then
2617 error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2618 FTPL_ERR=1
2619 fi
2620
2621 separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
2622 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2623
2624 done
2625
2626 exit ${FTPL_ERR}
