Sourcecode / SVN
Jump to navigation
Jump to search
Contents
duply sourcecode
duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.
SVN access
Browse the repository
https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/
SVN access
svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply
SVN Log
https://sourceforge.net/p/ftplicity/code/commit_browser
Latest Development Snapshot
mod time 2022-04-06
plain/text -> duply.sh
1 #!/usr/bin/env bash
2 #
3 ################################################################################
4 # duply (grown out of ftplicity), is a shell front end to duplicity that #
5 # simplifies the usage by managing settings for backup jobs in profiles. #
6 # It supports executing multiple commands in a batch mode to enable single #
7 # line cron entries and executes pre/post backup scripts. #
8 # Since version 1.5.0 all duplicity backends are supported. Hence the name #
9 # changed from ftplicity to duply. #
10 # See http://duply.net or http://ftplicity.sourceforge.net/ for more info. #
11 # (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany #
12 # (c) 2008-2022 Edgar Soldin (changes since version 1.3) #
13 ################################################################################
14 # LICENSE: #
15 # This program is licensed under GPLv2. #
16 # Please read the accompanying license information in gpl-2.0.txt. #
17 ################################################################################
18 # TODO/IDEAS/KNOWN PROBLEMS:
19 # - possibility to restore time frames (incl. deleted files)
20 # realizable by listing each backup and restore from
21 # oldest to the newest, problem: not performant
22 # - search file in all backups function and show available
23 # versions with backups date (list old avail since 0.6.06)
24 # - edit profile opens conf file in vi
25 # - implement log-fd interpretation
26 # - add a duplicity option check against the options pending
27 # deprecation since 0.5.10 namely --time-separator
28 # --short-filenames
29 # --old-filenames
30 # - add 'exclude_<command>' list usage e.g. exclude_verify
31 # - featreq 25: a download/install duplicity option
32 # - import/export profile from/to .tgz function !!!
33 # - remove url_encode, test for invalid chars n throw error instead
34 #
35 # CHANGELOG:
36 # 2.4 (6.4.2022)
37 # - bugfix #127: date_from_nsecs ignores format string
38 # - bugfix #116: separators print date now too
39 # - featreq #48: add purgeAuto command (see man page)
40 # - replaced tab indents with 2spaces everywhere
41 # - bugfix #129,131,132: duply stumbles over 'python -s' shebang,
42 # python interpreter parse failed if duplicity is a snap app
43 # - bugfix #130: duplicity version check failed "gpg: WARNING: ..."
44 # - version output, always print PYTHONPATH, if interpreter was determined
45 # - update python references to python3
46 #
47 # 2.3.1 (11.2.2021)
48 # - bugfix 123: symmetric encryption errs out, asks for '' private key
49 #
50 # 2.3 (30.12.2020)
51 # - don't import whole key pair anymore if only pub/sec is requested
52 # - gpg import routine informs on missing key files in profile now
53 # - add check/import needed secret key for decryption
54 # - featreq 50: Disable GPG key backups, implemented/added settings
55 # GPG_IMPORT/GPG_EXPORT='disabled' to conf template
56 #
57 # 2.2.2 (24.02.2020)
58 # - bugfix 120: Failures in "Autoset trust of key" during restore
59 # because of gpg2.2 fingerprint output change
60 #
61 # 2.2.1 (22.01.2020)
62 # - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
63 # - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
64 # - fix abbreviation spelling of 'e.g.'
65 #
66 # 2.2 (30.12.2018)
67 # - featreq 44: implement grouping for batch commands
68 # new separators are [] (square brackets) or groupIn/groupOut
69 # command 'backup' translates now to [pre_bkp_post] to be skipped as
70 # one block in case a condition was set in the batch instruction
71 #
72 # 2.1 (23.07.2018)
73 # - be more verbose when duplicity version detection fails
74 # - using info shows python binary's path for easier identification now
75 # - reworked python interpreter handling, it's either
76 # configured per PYTHON var
77 # unconfigured, parsed from duplicity shebang
78 # or set to current duplicity default 'python2' (was 'python' until now)
79 # - do not quotewrap strings because of slashes (e.g. paths) anymore
80 # - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
81 #
82 # 2.0.4 (20.02.2018)
83 # - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
84 # - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
85 # - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
86 # - bugfix debian 882159: duply: occasionally shows negative runtimes
87 #
88 # 2.0.3 (29.08.2017)
89 # - bugfix: "line 2231: CMDS: bad array subscript"
90 # - bugfix 112: "env: illegal option -- u" on MacOSX
91 #
92 # 2.0.2 (23.05.2017)
93 # - bugfix: never insert creds into file:// targets
94 # - bugfix: avail profiles hint sometimes shortend the names by one char
95 # - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
96 # - export condition before/after next/prev command as CND_PREV,CND_NEXT now
97 # - bugfix 97: Unknown command should be ERROR, not WARNING
98 #
99 # 2.0.1 (16.11.2016)
100 # - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
101 #
102 # 2.0 (27.10.2016)
103 # made this a major version change, as we broke backward compatibility anyway
104 # (see last change in v1.10). got complaints that rightfully pointed out
105 # that should only come w/ a major version change. so, here we go ;)
106 # if your backend stops working w/ this version create a new profile and
107 # export the env vars needed as described in the comments of the conf file
108 # directly above the SOURCE setting.
109 # Changes:
110 # - making sure multi spaces in TARGET survive awk processing
111 # - new env var PROFILE exported to scripts
112 # - fix 102: expose a unique timestamp variable for pre/post scripts
113 # actually a featreq. exporting RUN_START nanosec unix timestamp
114 # - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
115 # - fix 96: duply cannot handle two consecutive spaces in paths
116 #
117 # 1.11.3 (29.5.2016)
118 # - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
119 #
120 # 1.11.2 (11.2.2016)
121 # - fix "gpg: unsafe" version print out
122 # - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
123 # - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional
124 # documentation about needed env vars to template conf file
125 #
126 # 1.11.1 (18.12.2015)
127 # - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
128 # - reverted duply's default PYTHON to 'python'
129 #
130 # 1.11 (24.11.2015)
131 # - remove obsolete --ssh-askpass routine
132 # - add PYTHON conf var to allow global override of used python interpreter
133 # - enforced usage of "python2" in PATH as default interpreter for internal
134 # use _and_ to run duplicity (setup.py changed the shebang to the fixed
135 # path /usr/bin/python until 0.7.05, which we circumvent this way)
136 # - featreq 36: support gpg-connect-agent as a means to detect if an agent is
137 # running (thx Thomas Harning Jr.), used gpg-agent for detection though
138 # - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
139 # - key export routine respects gpg-agent usage now
140 #
141 # 1.10.1 (19.8.2015)
142 # - bugfix 86: Duply+Swift outputs warning
143 # - bugfix 87: Swift fails without BACKEND_URL
144 #
145 # 1.10 (31.7.2015)
146 # - featreq 37: busybox issues - fix awk, grep version detection,
147 # fix grep failure because --color=never switch is unsupported
148 # (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
149 # - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
150 # (thx Joachim Wiedorn, also for maintaining the debian package)
151 # - implemented base-/dirname as bash functions
152 # - featreq 31 " Support for duplicity Azure backend " - ignored a
153 # contributed patch by Scott McKenzie and instead opted for removing almost
154 # all code that deals with special env vars required by backends.
155 # adding and modifying these results in too much overhead so i dropped this
156 # feature. the future alternative for users is to consult the duplicity
157 # manpage and add the needed export definitions to the conf file.
158 # appended a commented example to the template conf below the auth section.
159 #
160 # 1.9.2 (21.6.2015)
161 # - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
162 # - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
163 # - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
164 # - added GPG conf var (see conf template for details)
165 # - added grep version output as it is an integral needed binary
166 # - added PYTHONPATH printout in version output
167 #
168 # 1.9.1 (13.10.2014)
169 # - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
170 # - bugfix: CMD_PREV contained command even if it was skipped
171 #
172 # 1.9.0 (24.8.2014)
173 # - bugfix: env vars were not exported when external script was executable
174 # - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.)
175 # see gpg manpage, section "How to specify a user ID"
176 # let gpg complain when the delivered values are invalid for whatever reason
177 # - started to rework tmp space checking, exposed folder & writable check
178 # TODO: reimplement enough file space available checking
179 #
180 # 1.8.0 (13.7.2014)
181 # - add command verifyPath to expose 'verify --file-to-restore' action
182 # - add time parameter support to verify command
183 # - add section time formats to usage output
184 #
185 # 1.7.4 (24.6.2014)
186 # - remove ubuntu one support, service is discontinued
187 # - featreq 31: add authenticated swift (contributed by Justus Seifert)
188 #
189 # 1.7.3 (3.4.2014)
190 # - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
191 #
192 # 1.7.2 (1.4.2014 "April,April")
193 # - bugfix: debian Bug#743190 "duply no longer allows restoration without
194 # gpg passphrase in conf file"
195 # GPG_AGENT_INFO env var is now needed to trigger --use-agent
196 # - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
197 #
198 # 1.7.1 (30.3.2014)
199 # - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to
200 # incompatibility with new minus batch separator
201 #
202 # 1.7.0 (20.3.2014)
203 # - disabled gpg key id plausibility check, too many valid possibilities
204 # - featreq 7 "Halt if precondition fails":
205 # added and(+), or(-) batch command(separator) support
206 # - featreq 26 "pre/post script with shebang line":
207 # if a script is flagged executable it's executed in a subshell
208 # now as opposed to sourced to bash, which is the default
209 # - bugfix: do not check if dpbx, swift credentials are set anymore
210 # - bugfix: properly escape profile name, archdir if used as arguments
211 # - add DUPL_PRECMD conf setting for use with e.g. trickle
212 #
213 # 1.6.0 (1.1.2014)
214 # - support gs backend
215 # - support dropbox backend
216 # - add gpg-agent support to gpg test routines
217 # - autoenable --use-agent if passwords were not defined in config
218 # - GPG_OPTS are now honored everywhere, keyrings or complete gpg
219 # homedir can thus be configured to be located anywhere
220 # - always import both secret and public key if avail from config profile
221 # - new explanatory comments in initial exclude file
222 # - bugfix 7: Duply only imports one key at a time
223 #
224 # 1.5.11 (19.07.2013)
225 # - purge-incr command for remove-all-inc-of-but-n-full feature added
226 # patch provided by Moritz Augsburger, thanks!
227 # - documented version command in man page
228 #
229 # 1.5.10 (26.03.2013)
230 # - minor indent and documentation fixes
231 # - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
232 # - bugfix: fix url_decoding generally and for python3
233 # - bugfix 3609075: wrong script results in status line (thx David Epping)
234 #
235 # 1.5.9 (22.11.2012)
236 # - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
237 # - restore/fetch now also ignores --include* or --exclude='foobar'
238 #
239 # 1.5.8 (26.10.2012)
240 # - bugfix 3575487: implement proper cloud files support
241 #
242 # 1.5.7 (10.06.2012)
243 # - bugfix 3531450: Cannot use space in target URL (file:///) anymore
244 #
245 # 1.5.6 (24.5.2012)
246 # - commands purge, purge-full have no default value anymore for security
247 # reasons; instead max value can be given via cmd line or must be set
248 # in profile; else an error is shown.
249 # - minor man page modifications
250 #
251 # versioning scheme will be simplified to [major].[minor].[patch] version
252 # with the next version raise
253 #
254 # 1.5.5.5 (4.2.2012)
255 # - bugfix 3479605: SEL context confused profile folder's permission check
256 # - colon ':' in url passphrase got ignored, added python driven url_decoding
257 # for user & pass to better process special chars
258 #
259 # 1.5.5.4 (16.10.2011)
260 # - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
261 # - add support for separate sign passphrase (needs duplicity 0.6.14+)
262 #
263 # 1.5.5.3 (1.10.2011)
264 # - bugfix 3416690: preview threw echo1 error
265 # - fix unknown cmds error usage & friends if more than 2 params were given
266 #
267 # 1.5.5.2 (23.9.2011)
268 # - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
269 # - bugfix: mawk does not support \W and did not split multikey definitions
270 # - all parameters should survive single (') and double (") quotes now
271 #
272 # 1.5.5.1 (7.6.2011)
273 # - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
274 # - bugfix 3312208: signing detection broke symmetric gpg test routine
275 #
276 # 1.5.5 (2.5.2011)
277 # - bugfix: fetch problem with space char in path, escape all params
278 # containing non word chars
279 # - list available profiles, if given profile cannot be found
280 # - added --use-agent configuration hint
281 # - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke
282 # fetch/restore
283 # - version command now prints out 'using installed' info
284 # - featreq 3166169: autotrust imported keys, based on code submitted by
285 # Martin Ellis - imported keys are now automagically trusted ultimately
286 # - new txt2man feature to create manpages for package maintainers
287 #
288 # 1.5.4.2 (6.1.2011)
289 # - new command changelog
290 # - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
291 # - bugfix: freebsd awk hangs on 'awk -W version'
292 # - bugfix 3150244: mawk does not know '--version'
293 # - minor help text improvements
294 # - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
295 #
296 # 1.5.4.1 (4.12.2010)
297 # - output awk, python, bash version now in prolog
298 # - shebang uses /usr/bin/env now for freebsd compatibility,
299 # bash not in /bin/bash
300 # - new --disable-encryption parameter,
301 # to override profile encr settings for one run
302 # - added exclude-if-present setting to conf template
303 # - bug 3126972: GPG_PW only needed for signing/symmetric encryption
304 # (even though duplicity still needs it)
305 #
306 # 1.5.4 (15.11.2010)
307 # - as of 1.5.3 already, new ARCH_DIR config option
308 # - multiple key support
309 # - ftplicity-Feature Requests-2994929: separate encryption and signing key
310 # - key signing of symmetric encryption possible (duplicity patch committed)
311 # - gpg tests disable switch
312 # - gpg tests now previewable and more intelligent
313 #
314 # 1.5.3 (1.11.2010)
315 # - bugfix 3056628: improve busybox compatibility, grep did not have -m param
316 # - bugfix 2995408: allow empty password for PGP key
317 # - bugfix 2996459: Duply erroneously escapes '-' symbol in username
318 # - url_encode function is now pythonized
319 # - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
320 # - feature 3059262: Make pre and post aware of parameters,
321 # internal parameters + CMD of pre or post
322 #
323 # 1.5.2.3 (16.4.2010)
324 # - bugfix: date again, should now work virtually anywhere
325 #
326 # 1.5.2.2 (3.4.2010)
327 # - minor bugfix: duplicity 0.6.8b version string now parsable
328 # - added INSTALL.txt
329 #
330 # 1.5.2.1 (23.3.2010)
331 # - bugfix: date formatting is awked now and should work on all platforms
332 #
333 # 1.5.2 (2.3.2010)
334 # - bugfix: errors print to STD_ERR now, failed tasks print an error message
335 # - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
336 # - simplified & cleaned profileless commands, removed second instance
337 # - generalized separator time routines
338 # - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
339 # - minor fixes
340 #
341 # 1.5.1.5 (5.2.2010)
342 # - bugfix: added special handling of credentials for rsync, imap(s)
343 #
344 # 1.5.1.4 (7.1.2010)
345 # - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
346 # - check if ncftp binary is available if url protocol is ftp
347 # - bugfix: duplicity output is now printed to screen directly to resolve
348 # 'mem alloc problem' bug report
349 # - bugfix: passwords will not be in the url anymore to solve the 'duply shows
350 # sensitive data in process listing' bug report
351 #
352 # 1.5.1.3 (24.12.2009) 'merry xmas'
353 # - bugfix: gpg pass now apostrophed to allow space and friends
354 # - bugfix: credentials are now url encoded to allow special chars in them
355 # a note about url encoding has been added to the conf template
356 #
357 # 1.5.1.2 (1.11.2009)
358 # - bugfix: open parenthesis in password broke duplicity execution
359 # - bugfix: ssh/scp backend does not always need credentials e.g. key auth
360 #
361 # 1.5.1.1 (21.09.2009)
362 # - bugfix: fixed s3[+http] TARGET_PASS not needed routine
363 # - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
364 # see https://sourceforge.net/tracker/index.php?func=detail&
365 # aid=2864410&group_id=217745&atid=1041147
366 #
367 # 1.5.1 (21.09.2009) - duply (fka. ftplicity)
368 # - first things first: ftplicity (being able to support all backends since
369 # some time) will be called duply (fka. ftplicity) from now on. The addendum
370 # is for the time being to circumvent confusion.
371 # - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity
372 # command failed
373 # - s3[+http] now supported natively by translating user/pass to access_key/
374 # secret_key environment variables needed by duplicity s3 boto backend
375 # - bugfix: additional output lines do not confuse version check anymore
376 # - list command supports now age parameter (patch by stefan on feature
377 # request tracker)
378 # - bugfix: option/param pairs are now correctly passed on to duplicity
379 # - bugfix: s3[+http] needs no TARGET_PASS if command is read only
380 #
381 # 1.5.0.2 (31.07.1009)
382 # - bugfix: insert password in target url didn't work with debian mawk
383 # related to previous bug report
384 #
385 # 1.5.0.1 (23.07.2009)
386 # - bugfix: gawk gensub dependency raised an error on debian's default mawk
387 # replaced with match/substr command combination (bug report)
388 # https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
389 # group_id=217745
390 #
391 # 1.5.0 (01.07.2009)
392 # - removed ftp limitation, all duplicity backends should work now
393 # - bugfix: date for separator failed on openwrt busybox date, added a
394 # detecting workaround, milliseconds are not available w/ busybox date
395 #
396 # 1.4.2.1 (14.05.2009)
397 # - bugfix: free temp space detection failed with lvm, fixed awk parse routine
398 #
399 # 1.4.2 (22.04.2009)
400 # - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
401 # armored ascii nature, the id helps if the key is switched for some reason
402 # im/export routines are updated accordingly (import is backward compatible
403 # to the old profile/gpgkey files)
404 # - profile argument is treated as path if it contains slashes
405 # (for details see usage)
406 # - non-ftplicity options (all but --preview currently) are now passed
407 # on to duplicity
408 # - removed need for stat in secure_conf, it is ls based now
409 # - added profile folder readable check
410 # - added gpg version & home info output
411 # - awk utility availability is now checked, because it was mandatory already
412 # - tmp space is now checked on writability and space requirement
413 # test fails on less than 25MB or configured $VOLSIZE,
414 # test warns if there is less than two times $VOLSIZE because
415 # that's required for --asynchronous-upload option
416 # - gpg functionality is tested now before executing duplicity
417 # test drive contains encryption, decryption, comparison, cleanup
418 # this is meant to detect non trusted or other gpg errors early
419 # - added possibility of doing symmetric encryption with duplicity
420 # set GPG_KEY="" or simply comment it out
421 # - added hints in config template on the depreciation of
422 # --short-filenames, --time-separator duplicity options
423 #
424 # new versioning scheme 1.4.2b => 1.4.2,
425 # beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
426 # to the first bug fixing version and 1.4.2.2 to the second and so on
427 # also the releases will now have a release date formatted (Day.Month.Year)
428 #
429 # 1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
430 # named exactly as the profile if existing in executing dir
431 # - improved plausibility checking of config and profile folder
432 # - secure_conf only acts if needed and prints a warning now
433 #
434 # 1.4.1b - introduce status (duplicity collection-status) command
435 # - pre/post script output printed always now, not only on errors
436 # - new config parameter GPG_OPTS to pass gpg options
437 # added examples & comments to profile template conf
438 # - reworked separator times, added duration display
439 # - added --preview switch, to preview generated command lines
440 # - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
441 # profiles because they have reasonable defaults now if not set
442 #
443 # 1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
444 # therefore backup translates to pre_bkp_post now
445 # - bugfix: new command bkp, which represents duplicity's
446 # default action (incr or full if full_if_older matches
447 # or no earlier backup chain is found)
448 #
449 # new versioning scheme 1.4 => 1.4.0, added new minor revision number
450 # this is meant to slow down the rapid version growing but still keep
451 # versions cleanly separated.
452 # only additional features will raise the new minor revision number.
453 # all releases start as beta, each bugfix release will raise the beta
454 # count, usually new features arrive before a version 'ripes' to stable
455 #
456 # 1.4.0b
457 # 1.4b - added startup info on version, time, selected profile
458 # - added time output to separation lines
459 # - introduced: command purge-full implements duplicity's
460 # remove-all-but-n-full functionality (patch by unknown),
461 # uses config variable $MAX_FULL_BACKUPS (default = 1)
462 # - purge config var $MAX_AGE defaults to 1M (month) now
463 # - command full does not execute pre/post anymore
464 # use batch command pre_full_post if needed
465 # - introduced batch mode cmd1_cmd2_etc
466 # (in turn removed the bvp command)
467 # - unknown/undefined command issues a warning/error now
468 # - bugfix: version check works with 0.4.2 and older now
469 # 1.3b3 - introduced pre/post commands to execute/debug scripts
470 # - introduced bvp (backup, verify, purge)
471 # - bugfix: removed need for awk gensub, now mawk compatible
472 # 1.3b2 - removed pre/post need executable bit set
473 # - profiles now under ~/.ftplicity as folders
474 # - root can keep profiles in /etc/ftplicity, folder must be
475 # created by hand, existing profiles must be moved there
476 # - removed ftplicity in path requirement
477 # - bugfix: bash < v.3 did not know '=~'
478 # - bugfix: purge works again
479 # 1.3 - introduces multiple profiles support
480 # - modified some script errors/docs
481 # - reordered gpg key check import routine
482 # - added 'gpg key id not set' check
483 # - added error_gpg (adds how to setup gpg key howto)
484 # - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
485 # - duplicity_version_check routine introduced
486 # - added time separator, shortnames, volsize, full_if_older
487 # duplicity options to config file (inspired by stevie
488 # from http://weareroot.de)
489 # 1.1.1 - bugfix: encryption reactivated
490 # 1.1 - introduced config directory
491 # 1.0 - first release
492 ################################################################################
493
494 # utility functions overriding binaries
495
496 # wrap grep to override possible env set GREP_OPTIONS=--color=always
497 function grep {
498 command env "GREP_OPTIONS=" grep "$@"
499 }
500
501 # implement basename in plain bash
502 function basename {
503 local stripped="${1%/}"
504 echo "${stripped##*/}"
505 }
506
507 # implement dirname in plain bash
508 function dirname {
509 echo ${1%/*}
510 }
511
512 # implement basic which in plain bash
513 function which {
514 type -p "$@"
515 }
516
517 # check availability of executables via file name or file paths
518 function lookup {
519 local bin="$1"
520 # look for file names in path via bash hash OR
521 # look for executables at given relative/absolute location
522 ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
523 }
524
525 # important definitions #######################################################
526
527 ME_LONG="$0"
528 ME="$(basename $0)"
529 ME_NAME="${ME%%.*}"
530 ME_VERSION="2.4"
531 ME_WEBSITE="https://duply.net"
532
533 # default config values
534 DEFAULT_SOURCE='/path/of/source'
535 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
536 DEFAULT_TARGET_USER='_backend_username_'
537 DEFAULT_TARGET_PASS='_backend_password_'
538 DEFAULT_GPG='gpg'
539 DEFAULT_GPG_KEY='_KEY_ID_'
540 DEFAULT_GPG_PW='_GPG_PASSWORD_'
541
542 # function definitions ##########################
543
544 function set_config { # sets global config vars
545 local CONFHOME_COMPAT="$HOME/.ftplicity"
546 local CONFHOME="$HOME/.duply"
547 local CONFHOME_ETC_COMPAT="/etc/ftplicity"
548 local CONFHOME_ETC="/etc/duply"
549
550 # confdir can be delivered as path (must contain /)
551 if [ `echo $FTPLCFG | grep /` ] ; then
552 CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
553 ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
554 && echo $(pwd)/${FTPLCFG} ) || \
555 echo ${FTPLCFG})
556 # or DEFAULT in home/.duply folder (NEW)
557 elif [ -d "${CONFHOME}" ]; then
558 CONFDIR="${CONFHOME}/${FTPLCFG}"
559 # or in home/.ftplicity folder (OLD)
560 elif [ -d "${CONFHOME_COMPAT}" ]; then
561 CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
562 warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
563 # root can put profiles under /etc/duply (NEW) if path exists
564 elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
565 CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
566 # root can keep profiles under /etc/ftplicity (OLD) if path exists
567 elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
568 CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
569 warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
570 # hmm no profile folder there, then use default for error later
571 else
572 CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
573 fi
574
575 # remove trailing slash, get profile name etc.
576 CONFDIR="${CONFDIR%/}"
577 PROFILE="${CONFDIR##*/}"
578 CONF="$CONFDIR/conf"
579 PRE="$CONFDIR/pre"
580 POST="$CONFDIR/post"
581 EXCLUDE="$CONFDIR/exclude"
582 KEYFILE="$CONFDIR/gpgkey.asc"
583 }
584
585 function version_info { # print version information
586 cat <<END
587 $ME_NAME version $ME_VERSION
588 ($ME_WEBSITE)
589 END
590 }
591
592 function version_info_using {
593 cat <<END
594 $(version_info)
595
596 $(using_info)
597 END
598 }
599
600 function using_info {
601 # init needed vars into global name space
602 lookup duplicity && { duplicity_version_get; }
603 local NOTFOUND="INVALID"
604 local AWK_VERSION GREP_VERSION PYTHON_RUNNER \
605 PYTHON_RUNNER_RESOLVED PYTHON_VERSION PYTHON_PATH
606 # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
607 AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
608 GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
609
610 if [ -n "$PYTHON" ]; then
611 PYTHON_RUNNER=$PYTHON
612 else
613 PYTHON_RUNNER="$(duplicity_python_binary_parse)"
614 fi
615 # fetch version and resolve python
616 [ -n "$PYTHON_RUNNER" ] && {
617 PYTHON_VERSION=$($PYTHON_RUNNER -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
618 local PYTHON_RUNNER_ARRAY=( $PYTHON_RUNNER )
619 PYTHON_RUNNER_RESOLVED="$(which ${PYTHON_RUNNER_ARRAY[0]})"
620 # readd params if there were
621 [ ${#PYTHON_RUNNER_ARRAY[@]} -gt 1 ] && \
622 PYTHON_RUNNER_RESOLVED="${PYTHON_RUNNER_RESOLVED} ${PYTHON_RUNNER_ARRAY[@]:1}"
623 PYTHON_PATH="$($PYTHON_RUNNER -c "import sys;print(':'.join(sys.path));")"
624 }
625
626 local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
627 local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
628 # print out
629 echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
630 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($PYTHON_RUNNER_RESOLVED)}${PYTHON_PATH:+ 'PYTHONPATH=$PYTHON_PATH'}}\
631 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
632 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
633 }
634
635 function usage_info { # print usage information
636
637 cat <<USAGE_EOF
638 VERSION:
639 $(version_info)
640
641 DESCRIPTION:
642 Duply deals as a wrapper for the mighty duplicity magic.
643 It simplifies running duplicity with cron or on command line by:
644
645 - keeping recurring settings in profiles per backup job
646 - enabling batch operations e.g. backup_verify+purge
647 - executing pre/post scripts (different actions possible
648 depending on previous or next command or it's exit status)
649 - precondition checking for flawless duplicity operation
650
651 For each backup job one configuration profile must be created.
652 The profile folder will be stored under '~/.${ME_NAME}/<profile>'
653 (where ~ is the current users home directory).
654 Hint:
655 If the folder '/etc/${ME_NAME}' exists, the profiles for the super
656 user root will be searched & created there.
657
658 USAGE:
659 first time usage (profile creation):
660 $ME <profile> create
661
662 general usage in single or batch mode (see EXAMPLES):
663 $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
664
665 For batches the conditional separators can also be written as pseudo commands
666 and(+), or(-). See SEPARATORS for details.
667
668 Non $ME options are passed on to duplicity (see OPTIONS).
669 All conf parameters can also be defined in the environment instead.
670
671 PROFILE:
672 Indicated by a path or a profile name (<profile>), which is resolved
673 to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
674
675 Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
676 the folder manually before running $ME_NAME as superuser.
677 Note:
678 Already existing profiles in root's home folder will cease to work
679 unless they are moved to the new location manually.
680
681 example 1: $ME humbug backup
682
683 Alternatively a _path_ might be used e.g. useful for quick testing,
684 restoring or exotic locations. Shell expansion should work as usual.
685 Hint:
686 The path must contain at least one path separator '/',
687 e.g. './test' instead of only 'test'.
688
689 example 2: $ME ~/.${ME_NAME}/humbug backup
690
691 SEPARATORS:
692 _ (underscore)
693 neutral separator
694 + (plus sign), _and_
695 conditional AND
696 the next command will only be executed if the previous succeeded
697 - (minus sign), _or_
698 conditional OR
699 the next command will only be executed if the previous failed
700 [] (square brackets), _groupIn_/_groupOut_
701 enables grouping of commands
702
703 example:
704 'pre+[bkp-verify]_post' translates to
705 'pre_and_groupIn_bkp_or_verify_groupOut_post'
706
707 COMMANDS:
708 usage get usage help text
709
710 and/or/groupIn/groupOut
711 pseudo commands used in batches (see SEPARATORS above)
712
713 create creates a configuration profile
714 backup backup with pre/post script execution (batch: [pre_bkp_post]),
715 full (if full_if_older matches or no earlier backup is found)
716 incremental (in all other cases)
717 pre/post execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
718 bkp as above but without executing pre/post scripts
719 full force full backup
720 incr force incremental backup
721 list [<age>]
722 list all files in backup (as it was at <age>, default: now)
723 status prints backup sets and chains currently in repository
724 verify [<age>] [--compare-data]
725 list files changed, since age if given
726 verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]
727 list changes of a file or folder path in backup compared to a
728 local path, since age if given
729 restore <target_path> [<age>]
730 restore the complete backup to <target_path> [as it was at <age>]
731 fetch <src_path> <target_path> [<age>]
732 fetch single file/folder from backup [as it was at <age>]
733 purge [<max_age>] [--force]
734 list outdated backup files (older than \$MAX_AGE)
735 [use --force to actually delete these files]
736 purgeFull [<max_full_backups>] [--force]
737 list outdated backup files (\$MAX_FULL_BACKUPS being the number of
738 full backups and associated incrementals to keep, counting in
739 reverse chronological order)
740 [use --force to actually delete these files]
741 purgeIncr [<max_fulls_with_incrs>] [--force]
742 list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being
743 the number of full backups which associated incrementals will be
744 kept, counting in reverse chronological order)
745 [use --force to actually delete these files]
746 purgeAuto [--force]
747 convenience batch wrapper for all purge commands above.
748 purge, purgeFull, purgeIncr are added if their conf vars were set. e.g.
749 MAX_AGE=1Y
750 MAX_FULL_BACKUPS=6
751 MAX_FULLS_WITH_INCR=3
752 in profile conf file would result in
753 [purge_purgeFull_purgeIncr]
754 cleanup [--force]
755 list broken backup chain files archives (e.g. after unfinished run)
756 [use --force to actually delete these files]
757
758 changelog print changelog / todo list
759 txt2man feature for package maintainers - create a manpage based on the
760 usage output. download txt2man from http://mvertes.free.fr/, put
761 it in the PATH and run '$ME txt2man' to create a man page.
762 version show version information of $ME_NAME and needed programs
763
764 OPTIONS:
765 --force passed to duplicity (see commands:
766 purge, purgeFull, purgeIncr, cleanup)
767 --preview do nothing but print out generated duplicity command lines
768 --disable-encryption
769 disable encryption, overrides profile settings
770
771 TIME FORMATS:
772 For all time related parameters like age, max_age etc.
773 Refer to the duplicity manpage for all available formats. Here some examples:
774 2002-01-25T07:00:00+02:00 (full date time format string)
775 2002/3/5 (date string YYYY/MM/DD)
776 12D (interval, 12 days ago)
777 1h78m (interval, 1 hour 78 minutes ago)
778
779 PRE/POST SCRIPTS:
780 Some useful internal duply variables are exported to the scripts.
781
782 PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>,
783 GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
784 CMD_<PREV|NEXT> (previous/next command),
785 CND_<PREV|NEXT> (condition before/after)
786
787 The CMD_* variables were introduced to allow different actions according to
788 the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post'
789 will call the pre script two times, with CMD_NEXT variable set to 'bkp'
790 on the first and to 'verify' on the second run.
791 CMD_ERR holds the exit code of the CMD_PREV .
792
793 EXAMPLES:
794 create profile 'humbug':
795 $ME humbug create (don't forget to edit this new conf file)
796 backup 'humbug' now:
797 $ME humbug backup
798 list available backup sets of profile 'humbug':
799 $ME humbug status
800 list and delete outdated backups of 'humbug':
801 $ME humbug purge --force
802 restore latest backup of 'humbug' to /mnt/restore:
803 $ME humbug restore /mnt/restore
804 restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:
805 $ME humbug fetch etc/passwd /root/pw 4D
806 (see "duplicity manpage", section TIME FORMATS)
807 a one line batch job on 'humbug' for cron execution:
808 $ME humbug backup_verify_purge --force
809 batch job to run a full backup with pre/post scripts:
810 $ME humbug pre_full_post
811
812 FILES:
813 in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
814 conf profile configuration file
815 pre,post pre/post scripts (see above for details)
816 gpgkey.*.asc exported GPG key files
817 exclude a globbing list of included or excluded files/folders
818 (see "duplicity manpage", section FILE SELECTION)
819
820 $(hint_profile)
821
822 SEE ALSO:
823 duplicity man page:
824 duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
825 USAGE_EOF
826 }
827
828 # to check call 'duply txt2man | man -l -'
829 function usage_txt2man {
830 usage_info | \
831 awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
832 txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
833 }
834
835 function changelog {
836 cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#( )?/,"",$0);print}'
837 }
838
839 function create_config {
840 if [ ! -d "$CONFDIR" ] ; then
841 mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
842 # create initial config file
843 cat <<EOF >"$CONF"
844 # gpg encryption settings, simple settings:
845 # GPG_KEY='disabled' - disables encryption alltogether
846 # GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
847 # sign if secret key of key1 is available use GPG_PW for sign & decrypt
848 # Note: you can specify keys via all methods described in gpg manpage,
849 # section "How to specify a user ID", escape commas (,) via backslash (\)
850 # e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
851 # as they are used to separate the entries
852 # GPG_PW='passphrase' - symmetric encryption using passphrase only
853 GPG_KEY='${DEFAULT_GPG_KEY}'
854 GPG_PW='${DEFAULT_GPG_PW}'
855 # gpg encryption settings in detail (extended settings)
856 # the above settings translate to the following more specific settings
857 # GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
858 # GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
859 # GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
860 # encryption. If you want to deliver different passphrases for e.g.
861 # several keys or symmetric encryption plus key signing you can use
862 # gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
863 # also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage
864 # notes on en/decryption
865 # private key and passphrase will only be needed for decryption or signing.
866 # decryption happens on restore and incrementals (compare archdir contents).
867 # for security reasons it makes sense to separate the signing key from the
868 # encryption keys. https://answers.launchpad.net/duplicity/+question/107216
869 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
870 #GPG_KEY_SIGN='<prvkey>'
871 # set if signing key passphrase differs from encryption (key) passphrase
872 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
873 #GPG_PW_SIGN='<signpass>'
874
875 # uncomment and set a file path or name force duply to use this gpg executable
876 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
877 #GPG='/usr/local/gpg-2.1/bin/gpg'
878
879 # gpg options passed from duplicity to gpg process (default='')
880 # e.g. "--trust-model pgp|classic|direct|always"
881 # or "--compress-algo=bzip2 --bzip2-compress-level=9"
882 # or "--personal-cipher-preferences AES256,AES192,AES..."
883 # or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
884 # or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
885 # also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
886 #GPG_OPTS=''
887
888 # disable preliminary tests with the following setting
889 #GPG_TEST='disabled'
890 # disable automatic gpg key importing altogether
891 #GPG_IMPORT='disabled'
892 # disable automatic gpg key exporting to profile folder
893 #GPG_EXPORT='disabled'
894
895 # backend, credentials & location of the backup target (URL-Format)
896 # generic syntax is
897 # scheme://[user[:password]@]host[:port]/[/]path
898 # e.g.
899 # sftp://bob:secret@backupserver.com//home/bob/dupbkp
900 # for details and available backends see duplicity manpage, section URL Format
901 # http://duplicity.nongnu.org/duplicity.1.html#sect7
902 # BE AWARE:
903 # some backends (cloudfiles, S3 etc.) need additional env vars to be set to
904 # work properly, read after the TARGET definition for more details.
905 # ATTENTION:
906 # characters other than A-Za-z0-9.-_.~ in the URL have to be
907 # replaced by their url encoded pendants, see
908 # http://en.wikipedia.org/wiki/Url_encoding
909 # if you define the credentials as TARGET_USER, TARGET_PASS below $ME
910 # will try to url_encode them for you if the need arises.
911 TARGET='${DEFAULT_TARGET}'
912
913 # optionally the username/password can be defined as extra variables
914 # setting them here _and_ in TARGET results in an error
915 # ATTENTION:
916 # there are backends that do not support the user/pass auth scheme.
917 # prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
918 # duplicity manpage. usually there is a NOTE section explaining if and which
919 # env vars should be set.
920 #TARGET_USER='${DEFAULT_TARGET_USER}'
921 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
922 # e.g. for cloud files backend it might look like this (uncomment for use!)
923 #export CLOUDFILES_USERNAME='someuser'
924 #export CLOUDFILES_APIKEY='somekey'
925 #export CLOUDFILES_AUTHURL ='someurl'
926 # the following is an incomplete list (<backend>: comma separated env vars list)
927 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
928 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
929 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
930 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
931 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
932 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
933 # SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
934
935 # base directory to backup
936 SOURCE='${DEFAULT_SOURCE}'
937
938 # a command that runs duplicity e.g.
939 # shape bandwidth use via trickle
940 # "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
941 #DUPL_PRECMD=""
942
943 # override the python interpreter to execute duplicity, unset by default
944 # e.g. "python3" or "/usr/bin/python3.8"
945 #PYTHON="python"
946
947 # exclude folders containing exclusion file (since duplicity 0.5.14)
948 # Uncomment the following two lines to enable this setting.
949 #FILENAME='.duplicity-ignore'
950 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
951
952 # Time frame for old backups to keep, Used for the "purge" command.
953 # see duplicity man page, chapter TIME_FORMATS)
954 #MAX_AGE=1M
955
956 # Number of full backups to keep. Used for the "purgeFull" command.
957 # See duplicity man page, action "remove-all-but-n-full".
958 #MAX_FULL_BACKUPS=1
959
960 # Number of full backups for which incrementals will be kept for.
961 # Used for the "purgeIncr" command.
962 # See duplicity man page, action "remove-all-inc-of-but-n-full".
963 #MAX_FULLS_WITH_INCRS=1
964
965 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3)
966 # forces a full backup if last full backup reaches a specified age, for the
967 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
968 # Uncomment the following two lines to enable this setting.
969 #MAX_FULLBKP_AGE=1M
970 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE "
971
972 # sets duplicity --volsize option (available since v0.4.3.RC7)
973 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
974 # VOLSIZE must be number of MB's to set the volume size to.
975 # Uncomment the following two lines to enable this setting.
976 #VOLSIZE=50
977 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
978
979 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
980 # default is 4, if not set
981 #VERBOSITY=5
982
983 # temporary file space. at least the size of the biggest file in backup
984 # for a successful restoration process. (default is '/tmp', if not set)
985 #TEMP_DIR=/tmp
986
987 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds
988 # unencrypted meta data of the backup, enabling new incrementals without the
989 # need to decrypt backend metadata first. If empty or deleted somehow, the
990 # private key and it's password are needed.
991 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite
992 # big over time so you might want to put it not in the home dir.
993 # default '~/.cache/duplicity/duply_<profile>/'
994 # if set '\${ARCH_DIR}/<profile>'
995 #ARCH_DIR=/some/space/safe/.duply-cache
996
997 # DEPRECATED setting
998 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users
999 # to change the time separator from ':' to another character that will work
1000 # on their system. HINT: For Windows SMB shares, use --time-separator='_'.
1001 # NOTE: '-' is not valid as it conflicts with date separator.
1002 # ATTENTION: only use this with duplicity < 0.5.10, since then default file
1003 # naming is compatible and this option is pending depreciation
1004 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
1005
1006 # DEPRECATED setting
1007 # activates duplicity --short-filenames option, when uploading to a file
1008 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
1009 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
1010 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file
1011 # naming is compatible and this option is pending depreciation
1012 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
1013
1014 # more duplicity command line options can be added in the following way
1015 # don't forget to leave a separating space char at the end
1016 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here "
1017
1018 EOF
1019
1020 # create initial exclude file
1021 cat <<EOF >"$EXCLUDE"
1022 # although called exclude, this file is actually a globbing file list
1023 # duplicity accepts some globbing patterns, even including ones here
1024 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1025 # - dir/bar/foo
1026 # + dir/bar
1027 # - **
1028 # for more details see duplicity manpage, section File Selection
1029 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1030
1031 EOF
1032
1033 # Hints on first usage
1034 cat <<EOF
1035
1036 Congratulations. You just created the profile '$FTPLCFG'.
1037 The initial config file has been created as
1038 '$CONF'.
1039 You should now adjust this config file to your needs.
1040
1041 $(hint_profile)
1042
1043 EOF
1044 fi
1045
1046 }
1047
1048 # used in usage AND create_config
1049 function hint_profile {
1050 cat <<EOF
1051 IMPORTANT:
1052 Copy the _whole_ profile folder after the first backup to a safe place.
1053 It contains everything (duply related) needed to restore your backups.
1054
1055 Pay attention to (possibly later added) external files such as credentials
1056 or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1057 (e.g. DPBX_ACCESS_TOKEN).
1058 It is good policy to place those in the profile folder if possible at all.
1059 e.g. in case of 'multi://' target the config .json file
1060 Env vars should be added to duply profiles' conf file.
1061
1062 Keep access to these files restricted as they contain information (gpg key,
1063 passphrases etc.) to access and modify your backups.
1064
1065 Finally:
1066 You should attempt a restore from an unrelated host to be sure you really
1067 have everything needed for restoration.
1068
1069 Repeat these steps after _all_ configuration changes. Some configuration
1070 options are crucial for restoration.
1071
1072 EOF
1073 }
1074
1075 function separator {
1076 echo "--- $@ ---"
1077 }
1078
1079 function inform {
1080 echo -e "\nINFO:\n\n$@\n"
1081 }
1082
1083 function warning {
1084 echo -e "\nWARNING:\n\n$@\n"
1085 }
1086
1087 function warning_oldhome {
1088 local old=$1 new=$2
1089 warning " ftplicity changed name to duply since you created your profiles.
1090 Please rename the old folder
1091 '$old'
1092 to
1093 '$new'
1094 and this warning will disappear.
1095 If you decide not to do so profiles will _only_ work from the old location."
1096 }
1097
1098 function error_print {
1099 echo -e "$@" >&2
1100 }
1101
1102 function error {
1103 error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1104 exit -1
1105 }
1106
1107 function error_gpg {
1108 [ -n "$2" ] && local hint="\n $2\n\n "
1109
1110 error "$1
1111
1112 Hint${hint:+s}:
1113 ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1114 Don't forget the used _password_ as you will need it.
1115 When done enter the 8 digit id & the password in the profile conf file.
1116
1117 The key id can be found doing a 'gpg --list-keys'. In the example output
1118 below the key id for the public key would be FFFFFFFF.
1119
1120 pub 1024D/FFFFFFFF 2007-12-17
1121 uid duplicity
1122 sub 2048g/899FE27F 2007-12-17
1123 "
1124 }
1125
1126 function error_gpg_test {
1127 [ -n "$2" ] && local hint="\n $2\n\n "
1128
1129 error "$1
1130
1131 Hint${hint:+s}:
1132 ${hint}This error means that gpg is probably misconfigured or not working
1133 correctly. The error message above should help to solve the problem.
1134 However, if for some reason $ME_NAME should misinterpret the situation you
1135 can define GPG_TEST='disabled' in the conf file to bypass the test.
1136 Please do not forget to report the bug in order to resolve the problem
1137 in future versions of $ME_NAME.
1138 "
1139 }
1140
1141 function error_path {
1142 error "$@
1143 PATH='$PATH'
1144 "
1145 }
1146
1147 function error_to_string {
1148 [ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1149 }
1150
1151 function duplicity_version_get {
1152 # use cached value, just print
1153 var_isset DUPL_VERSION && return
1154
1155 local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1156 # only run with a user specific python if configured (running by default
1157 # breaks homebrew as they place a shell wrapper for duplicity in path)
1158 [ -n "$PYTHON" ] &&\
1159 CMD="$PYTHON $(qw "$(which duplicity)")"
1160
1161 DUPL_VERSION_OUT=$($CMD --version)
1162 DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1163 #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1164 DUPL_VERSION_VALUE=0
1165 DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1166 if (match(v,/[^\.0-9]+[0-9]*$/)){
1167 rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1168 if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1169 split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1170 print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1171 print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1172 print "DUPL_VERSION_RC=\047"rc"\047"
1173 print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1174 }')
1175 eval "$DUPL_VERSION_AWK"
1176 #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1177
1178 # doublecheck findings and report error
1179 if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1180 inform "duplicity version check failed (please report, this is a bug)
1181 the command
1182 $CMD
1183 resulted in
1184 $DUPL_VERSION_OUT
1185 "
1186 elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1187 error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1188 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1189 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1190 fi
1191 }
1192
1193 function duplicity_version_ge {
1194 [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1195 }
1196
1197 function duplicity_version_lt {
1198 ! duplicity_version_ge "$1"
1199 }
1200
1201 # parse interpreter from duplicity shebang
1202 function duplicity_python_binary_parse {
1203 # reuse cached result
1204 var_isset 'DUPL_PYTHON_BIN' && {
1205 [ -n "$DUPL_PYTHON_BIN" ] && {
1206 echo $DUPL_PYTHON_BIN
1207 return
1208 } || return 1
1209 }
1210
1211 local DUPL_BIN=$(which duplicity)
1212 # test for shebang
1213 IFS= LC_ALL=C read -rN2 shebang < "$DUPL_BIN" && [ "$shebang" != '#!' ] && {
1214 DUPL_PYTHON_BIN=""
1215 return 1
1216 }
1217
1218 # parse it or warn
1219 DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1220 if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1221 warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was
1222 '$DUPL_PYTHON_BIN'.
1223 "
1224 DUPL_PYTHON_BIN=""
1225 return 1
1226 fi
1227
1228 # success
1229 echo $DUPL_PYTHON_BIN
1230 return
1231 }
1232
1233 function run_script { # run pre/post scripts
1234 local ERR=0
1235 local SCRIPT="$1"
1236 if [ ! -z "$PREVIEW" ] ; then
1237 echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1238 elif [ -r "$SCRIPT" ] ; then
1239 echo -n "Running '$SCRIPT' "
1240 if [ -x "$SCRIPT" ]; then
1241 OUT=$("$SCRIPT" 2>&1)
1242 ERR=$?
1243 else
1244 OUT=$(. "$SCRIPT" 2>&1)
1245 ERR=$?
1246 fi
1247 [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1248 echo -en ${OUT:+"Output: $OUT\n"} ;
1249 else
1250 echo "Skipping n/a script '$SCRIPT'."
1251 fi
1252 return $ERR
1253 }
1254
1255 function run_cmd {
1256 # run or print escaped cmd string
1257 local CMD_ERR=0
1258 if [ -n "$PREVIEW" ]; then
1259 CMD_OUT=$( echo "$@ 2>&1" )
1260 CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1261 elif [ -n "$CMD_DISABLED" ]; then
1262 CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1263 else
1264 echo -n -e "$CMD_MSG"
1265 CMD_OUT=` eval "$@" 2>&1 `
1266 CMD_ERR=$?
1267 if [ "$CMD_ERR" = "0" ]; then
1268 CMD_MSG=" (OK)"
1269 else
1270 CMD_MSG=" (FAILED)"
1271 fi
1272 fi
1273 echo -e "$CMD_MSG"
1274 # reset
1275 unset CMD_DISABLED CMD_MSG
1276 return $CMD_ERR
1277 }
1278
1279 function qw { quotewrap "$@"; }
1280
1281 function quotewrap {
1282 local param="$@"
1283 # quote strings having non word chars (e.g. spaces)
1284 if echo "$param" | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1285 echo "$param" | awk '{\
1286 gsub(/[\047]/,"\047\\\047\047",$0);\
1287 gsub(/[\042]/,"\047\\\042\047",$0);\
1288 print "\047"$0"\047"}'
1289 return
1290 fi
1291 echo $param
1292 }
1293
1294 function duplicity_params_global {
1295 # already done? return
1296 var_isset 'DUPL_PARAMS_GLOBAL' && return
1297 local DUPL_ARG_ENC
1298
1299 # use key only if set in config, else leave it to symmetric encryption
1300 if gpg_disabled; then
1301 local DUPL_PARAM_ENC='--no-encryption'
1302 else
1303 local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1304 gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1305 # interpret password settings
1306 var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1307 var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1308 fi
1309
1310 local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1311
1312 # set name for dupl archive folder, since 0.6.0
1313 if duplicity_version_ge 601; then
1314 local DUPL_ARCHDIR=''
1315 if var_isset 'ARCH_DIR'; then
1316 DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1317 # reuse erronously duply_ prefixed folders from bug #117
1318 if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1319 DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1320 else
1321 DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1322 fi
1323 else
1324 DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1325 fi
1326 fi
1327
1328 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1329 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1330 ${GPG_OPTS}"
1331
1332 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1333 ${DUPL_ARG_ENC}"
1334 }
1335
1336 # function to filter the DUPL_PARAMS var from user conf
1337 function duplicity_params_conf {
1338 # reuse cmd var from main loop
1339 ## in/exclude parameters are currently not supported on restores
1340 if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1341 # filter exclude params from fetch/restore/status
1342 eval "stripXcludes $DUPL_PARAMS"
1343 return
1344 fi
1345
1346 # nothing done, print unchanged
1347 echo "$DUPL_PARAMS"
1348 }
1349
1350 # strip in/exclude parameters from param string
1351 function stripXcludes {
1352 local STRIPNEXT OUT;
1353 for p in "$@"; do
1354 if [ -n "$STRIPNEXT" ]; then
1355 unset STRIPNEXT
1356 # strip the value of previous parameter
1357 continue
1358 elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1359 # strips e.g. --include /foo/bar
1360 STRIPNEXT="yes"
1361 continue
1362 elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1363 # strips e.g. --include=/foo/bar
1364 continue
1365 fi
1366
1367 OUT="$OUT $(qw "$p")"
1368 done
1369 echo "$OUT"
1370 }
1371
1372 function duplify { # the actual wrapper function
1373 local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1374
1375 # put command (with params) first in duplicity parameters
1376 for param in "$@" ; do
1377 # split cmd from params (everything before splitchar --)
1378 if [ "$param" == "--" ] ; then
1379 PARAMSNOW=1
1380 else
1381 # wrap in quotes to protect from spaces
1382 [ ! $PARAMSNOW ] && \
1383 DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1384 || \
1385 DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1386 fi
1387 done
1388
1389 # init global duplicity parameters same for all tasks
1390 duplicity_params_global
1391
1392 local RUN=eval CMD=duplicity
1393 # run in cmd line preview mode if requested
1394 var_isset 'PREVIEW' && RUN=echo
1395 # only run with a user specific python if configured (running by default
1396 # breaks homebrew as they place a shell wrapper for duplicity in path)
1397 # resolve duplicity path for usage with python interpreter
1398 [ -n "$PYTHON" ] &&\
1399 CMD="$PYTHON $(qw "$(which duplicity)")"
1400
1401 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1402 ${DUPL_PRECMD} $CMD $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1403 $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1404
1405 local ERR=$?
1406 return $ERR
1407 }
1408
1409 function secureconf { # secure the configuration dir
1410 #PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1411 local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1412 if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1413 chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1414 warning "The profile's folder
1415 '$CONFDIR'
1416 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1417 fi
1418 }
1419
1420 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1421 function date_fix {
1422 local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1423 local date
1424 #[ "$1" == "%N" ] && return #test the no nsec test below
1425 # gnu date with -d @epoch
1426 date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1427 echo $date && return
1428 # date bsd,osx with -r epoch
1429 date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1430 echo $date && return
1431 # date busybox with -d epoch -D %s
1432 date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1433 echo $date && return
1434 ## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1435 # python fallback
1436 #date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1437 # echo $date && return
1438 # awk fallback
1439 date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1440 echo $date && return
1441 # perl fallback
1442 date=$(perl -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1443 echo $date && return
1444 # error
1445 echo "ERROR"
1446 return 1
1447 }
1448
1449 function nsecs {
1450 local NSECS
1451 # test if date supports nanosecond output
1452 if ! var_isset NSECS_DISABLED; then
1453 NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1454 [ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1455 fi
1456
1457 # add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1458 if [ "$NSECS_DISABLED" == "1" ]; then
1459 date_fix %s000000000
1460 else
1461 date_fix %s%N
1462 fi
1463 }
1464
1465 function nsecs_to_sec {
1466 echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1467 }
1468
1469 function datefull_from_nsecs {
1470 date_from_nsecs $1 '%F %T'
1471 }
1472
1473 function date_from_nsecs {
1474 local FORMAT=${2:-%T}
1475 local TIME=$(nsecs_to_sec $1)
1476 local SECS=${TIME%.*}
1477 local DATE=$(date_fix "${FORMAT}" ${SECS:-0})
1478 echo $DATE.${TIME#*.}
1479 }
1480
1481 function var_isset {
1482 if [ -z "$1" ]; then
1483 echo "ERROR: function var_isset needs a string as parameter"
1484 elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1485 return 0
1486 fi
1487 return 1
1488 }
1489
1490 function is_condition {
1491 local CMD=$(tolower "$1")
1492 [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1493 }
1494
1495 function is_groupMarker {
1496 local CMD=$(tolower "$1")
1497 [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1498 }
1499
1500 function is_command {
1501 local CMD=$(tolower "$1")
1502 ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1503 }
1504
1505 function url_encode {
1506 # utilize python, silently do nothing on error - because no python no duplicity
1507 local PYTHON_RUNNER
1508 PYTHON_RUNNER="$(duplicity_python_binary_parse)" ||\
1509 PYTHON_RUNNER="python" &&\
1510 OUT=$($PYTHON_RUNNER -c "
1511 try: import urllib.request as urllib
1512 except ImportError: import urllib
1513 print(urllib.${2}quote('$1'));
1514 " 2>/dev/null ); ERR=$?
1515 [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1516 }
1517
1518 function url_decode {
1519 # reuse function above with a simple string param hack
1520 url_encode "$1" "un"
1521 }
1522
1523 function toupper {
1524 echo "$@"|awk '$0=toupper($0)'
1525 }
1526
1527 function tolower {
1528 echo "$@"|awk '$0=tolower($0)'
1529 }
1530
1531 function isnumber {
1532 case "$*" in
1533 ''|*[!0-9]*) return 1;;
1534 *) return 0;;
1535 esac
1536 }
1537
1538 #function tmp_space {
1539 #
1540 # if ! isnumber $VOLSIZE; then
1541 # inform "failed to determine free space (please report, this is a bug)"
1542 # return
1543 # fi
1544 #
1545 # get free temp space
1546 # TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1547 # # check for free space or FAIL
1548 # if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1549 # error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1550 #than one duplicity volume (${VOLSIZE}MB).
1551 #
1552 # Hint: Free space or change TEMP_DIR setting."
1553 #fi
1554 #
1555 #}
1556
1557 function gpg_disabled {
1558 echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1559 }
1560
1561 # usage: join SEPARATOR "entry1" "entry2"
1562 function join {
1563 local SEP="$1" ENTRY OUT; shift;
1564 for ENTRY in "$@"; do
1565 ENTRY=${ENTRY//$SEP/\\$SEP}
1566 [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1567 done
1568 echo $OUT
1569 }
1570
1571 function gpg_testing {
1572 [ "$GPG_TEST" != "disabled" ]
1573 }
1574
1575 function gpg_signing {
1576 echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1577 }
1578
1579 function gpg_keytype {
1580 echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1581 }
1582
1583 # parameter key id, key_type
1584 function gpg_keyfile {
1585 local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1586 local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1587 echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1588 }
1589
1590 # parameter key id
1591 function gpg_import {
1592 local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1593 [ "$GPG_IMPORT" = "disabled" ] && {
1594 echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1595 return
1596 }
1597
1598 # create a list of legacy key file names and current naming scheme
1599 # we always import pub and sec if they are avail in conf folder
1600 local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1601 $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1602
1603 # Try autoimport from existing old gpgkey files
1604 # and new gpgkey.XXX.asc files (since v1.4.2)
1605 # and even newer gpgkey.XXX.[pub|sec].asc
1606 for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1607 FILE=${KEYFILES[$i]}
1608 if [ -f "$FILE" ]; then
1609 FOUND=1
1610
1611 CMD_MSG="Import keyfile '$FILE' to keyring"
1612 run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1613 if [ "$?" != "0" ]; then
1614 warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1615 ERR=1
1616 # continue with next
1617 continue
1618 fi
1619 fi
1620 done
1621
1622 if [ "$FOUND" -eq 0 ]; then
1623 echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1624 return 1
1625 fi
1626
1627 # try to set trust automagically
1628 CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1629 run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1630 if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then
1631 # success on all levels, we're done
1632 return $ERR
1633 fi
1634
1635 # failover: user has to set trust manually
1636 echo -e "For $ME_NAME to work you have to set the trust level
1637 with the command \"trust\" to \"ultimate\" (5) now.
1638 Exit the edit mode of gpg with \"quit\"."
1639 CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1640 run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1641
1642 return $ERR
1643 }
1644
1645 # see 'How to specify a user ID' on gpg manpage
1646 function gpg_fingerprint {
1647 gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1648 awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1649 }
1650
1651 function gpg_export_if_needed {
1652 [ "$GPG_EXPORT" = 'disabled' ] && { \
1653 echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1654 return
1655 }
1656
1657 local SUCCESS FILE KEY_TYPE
1658 local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1659 for KEY_ID in "$@"; do
1660 # check if already exported, do it if not
1661 for KEY_TYPE in PUB SEC; do
1662 FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1663 if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1664
1665 # exporting
1666 CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1667 # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1668 # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1669 run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1670 CMD_ERR=$?
1671
1672 if [ "$CMD_ERR" = "0" ]; then
1673 CMD_MSG="Write file '"$(basename "$FILE")"'"
1674 run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1675 fi
1676
1677 if [ "$CMD_ERR" != "0" ]; then
1678 warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1679 else
1680 SUCCESS=1
1681 fi
1682
1683 # cleanup
1684 rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1685 fi
1686 done
1687 done
1688
1689 [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1690 You should backup your changed profile folder now and store it in a safe place."
1691 }
1692
1693 # replace all non-alnum chars with underscore (for file operations)
1694 function gpg_key_legalize {
1695 echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1696 }
1697
1698 function gpg_key_cache {
1699 local RES
1700 local MODE=$1
1701 shift
1702 local PREFIX="GPG_KEY"
1703 local SUFFIX=$(gpg_key_legalize "$@")
1704 local KEYID="$*"
1705 local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1706 if [ "$MODE" = "RESET" ]; then
1707 eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1708 return 255
1709 elif ! var_isset "$CACHE"; then
1710 if [ "$MODE" = "PUB" ]; then
1711 RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1712 elif [ "$MODE" = "SEC" ]; then
1713 RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1714 else
1715 return 255
1716 fi
1717 eval $CACHE=$RES
1718 fi
1719 eval return \$$CACHE
1720 }
1721
1722 function gpg_pub_avail {
1723 gpg_key_cache PUB "$@"
1724 }
1725
1726 function gpg_sec_avail {
1727 gpg_key_cache SEC "$@"
1728 }
1729
1730 function gpg_key_format {
1731 echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1732 }
1733
1734 # splits a comma separated line into lines, respects escaped commas
1735 function gpg_split_keyset {
1736 local LIST
1737 LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1738 echo -e "$LIST"
1739 }
1740
1741 function gpg_prefix_keyset {
1742 local PREFIX="$1" OUT=""
1743 shift
1744 for KEY_ID in "$@"; do
1745 OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1746 done
1747 echo $OUT
1748 }
1749
1750 # grep a variable from conf text file (currently not used)
1751 function gpg_passwd {
1752 [ -r "$CONF" ] && \
1753 awk '/^[ \t]*GPG_PW[ \t=]/{\
1754 sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1755 gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1756 print $0; exit}' "$CONF"
1757 }
1758
1759 # return success if at least one secret key is available
1760 function gpg_key_decryptable {
1761 # no keys, no problem
1762 gpg_symmetric && return 0
1763
1764 local KEY_ID
1765 for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1766 gpg_sec_avail "$KEY_ID" && return 0
1767 done
1768 return 1
1769 }
1770
1771 function gpg_symmetric {
1772 [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1773 }
1774
1775 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1776 function gpg_param_passwd {
1777 var_isset GPG_USEAGENT && exit 1
1778
1779 if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1780 echo "--passphrase-fd 0 --batch"
1781 fi
1782 }
1783
1784 # select the earliest defined and create an "echo <value> |" string
1785 function gpg_pass_pipein {
1786 var_isset GPG_USEAGENT && exit 1
1787
1788 for var in "$@"
1789 do
1790 if var_isset "$var"; then
1791 echo "echo $(qw $(eval echo \$$var)) |"
1792 return 0
1793 fi
1794 done
1795
1796 return 1
1797 }
1798
1799 # checks if gpg-agent is available, returns error code
1800 # 0 on success
1801 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1802 # 2 if GPG_AGENT_INFO is stale
1803 # 3 cannot connect to gpg-agent
1804 function gpg_agent_avail {
1805 # GPG_AGENT_INFO is deprecated in gpg2.1,
1806 # first try to connect to a possibly running agent here
1807 local ERR=3
1808 gpg-agent > /dev/null 2>&1 && return 0
1809
1810 # detect stale pid in legacy GPG_AGENT_INFO env var
1811 if var_isset GPG_AGENT_INFO; then
1812 # check if a pid matching process is running at all
1813 local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1814 if isnumber "$GPG_AGENT_PID"; then
1815 ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1816 fi
1817 fi
1818
1819 return $ERR
1820 }
1821
1822 function gpg_custom_binary {
1823 var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1824 echo "--gpg-binary $(qw "$GPG")"
1825 }
1826
1827 function gpg_binary {
1828 local BIN
1829 var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1830 echo "$BIN"
1831 }
1832
1833 function gpg_avail {
1834 lookup "$(gpg_binary)"
1835 }
1836
1837 # enforce the use of our selected gpg binary
1838 function gpg {
1839 command "$(gpg_binary)" "$@"
1840 }
1841 export -f gpg
1842
1843 # start of script #######################################################################
1844
1845 # confidentiality first, all we create is only readable by us
1846 umask 077
1847
1848 # check if ftplicity is there & executable
1849 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1850
1851 if [ ${#@} -eq 1 ]; then
1852 cmd="${1}"
1853 else
1854 FTPLCFG="${1}" ; cmd="${2}"
1855 fi
1856
1857 # deal with command before profile validation calls
1858 # show requested version
1859 # OR requested usage info
1860 # OR create a profile
1861 # OR fall through
1862 ##if [ ${#@} -le 2 ]; then
1863 case "$cmd" in
1864 changelog)
1865 changelog
1866 exit 0
1867 ;;
1868 create)
1869 set_config
1870 if [ -d "$CONFDIR" ]; then
1871 error "The profile '$FTPLCFG' already exists in
1872 '$CONFDIR'.
1873
1874 Hint:
1875 If you _really_ want to create a new profile by this name you will
1876 have to manually delete the existing profile folder first."
1877 exit 1
1878 else
1879 create_config
1880 exit 0
1881 fi
1882 ;;
1883 txt2man)
1884 set_config
1885 usage_txt2man
1886 exit 0
1887 ;;
1888 usage|-help|--help|-h|-H)
1889 set_config
1890 usage_info
1891 exit 0
1892 ;;
1893 version|-version|--version|-v|-V)
1894 # profile can override GPG/PYTHON, so import it if it was given
1895 var_isset FTPLCFG && {
1896 set_config
1897 [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1898 }
1899 version_info_using
1900 exit 0
1901 ;;
1902 # fallthrough.. we got a command that needs an existing profile
1903 *)
1904 # if we reach here, user either forgot profile or chose wrong profileless command
1905 if [ ${#@} -le 1 ]; then
1906 error "\
1907 Missing or wrong parameters.
1908 Only the commands
1909 changelog, create, usage, txt2man, version
1910 can be called without selecting an existing profile first.
1911 Your command was '$cmd'.
1912
1913 Hint: Run '$ME usage' to get help."
1914 fi
1915 esac
1916
1917
1918 # Hello world
1919 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1920
1921 # check system environment
1922
1923 # is duplicity avail
1924 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1925 # init, exec duplicity version check info
1926 duplicity_version_get
1927
1928 # check for certain important helper programs
1929 # TODO: we should probably check for duplicity and $PYTHON (if set) here too
1930 for f in awk grep ; do
1931 lookup "$f" || \
1932 error_path "$f missing. installed und available in path?"
1933 done
1934
1935 ### read configuration
1936 set_config
1937 # check validity
1938 if [ ! -d "$CONFDIR" ]; then
1939 error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1940 '$CONFDIR'.
1941
1942 Hints:
1943 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1944 Use '$ME <name> create' to create a new profile.
1945 Use '$ME usage' to get usage help."
1946 elif [ ! -x "$CONFDIR" ]; then
1947 error "\
1948 Profile folder in '$CONFDIR' cannot be accessed.
1949
1950 Hint:
1951 Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1952 elif [ ! -f "$CONF" ] ; then
1953 error "'$CONF' not found."
1954 elif [ ! -r "$CONF" ] ; then
1955 error "'$CONF' not readable."
1956 else
1957 . "$CONF"
1958 #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1959 TEMP_DIR=${TEMP_DIR:-'/tmp'}
1960 # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1961 if var_isset 'TARGET_PW'; then
1962 TARGET_PASS="${TARGET_PW}"
1963 fi
1964 fi
1965 echo "Using profile '$CONFDIR'."
1966
1967 # secure config dir, if needed w/ warning
1968 secureconf
1969
1970 # split TARGET in handy variables
1971 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1972 target=$0; match(target,/^([^\/:]+):\/\//); \
1973 prot=substr(target,RSTART,RLENGTH);\
1974 rest=substr(target,RSTART+RLENGTH); \
1975 if (credsavail=match(rest,/^[^@]*@/)){\
1976 creds=substr(rest,RSTART,RLENGTH-1);\
1977 credcount=split(creds,cred,":");\
1978 rest=substr(rest,RLENGTH+1);\
1979 # split creds with regexp\
1980 match(creds,/^([^:]+)/);\
1981 user=substr(creds,RSTART,RLENGTH);\
1982 pass=substr(creds,RSTART+1+RLENGTH);\
1983 };\
1984 # filter quotes or escape them\
1985 gsub(/[\047\042]/,"",prot);\
1986 gsub(/[\047\042]/,"",rest);\
1987 gsub(/[\047]/,"\047\\\047\047",creds);\
1988 print "TARGET_URL_PROT=\047"prot"\047\n"\
1989 "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1990 "TARGET_URL_CREDS=\047"creds"\047\n";\
1991 if(user){\
1992 gsub(/[\047]/,"\047\\\047\047",user);\
1993 print "TARGET_URL_USER=\047"user"\047\n"}\
1994 if(pass){\
1995 gsub(/[\047]/,"\047\\\047\047",pass);\
1996 print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1997 }')
1998 eval "${TARGET_SPLIT_URL}"
1999
2000 # fetch commmand from parameters ########################################################
2001 # Hint: cmds is also used to check if authentification info sufficient in the next step
2002 cmds="$2"; shift 2
2003
2004 # complain if command(s) missing
2005 [ -z $cmds ] && error " No command given.
2006
2007 Hint:
2008 Use '$ME usage' to get usage help."
2009
2010 # process params
2011 for param in "$@"; do
2012 #echo !$param!
2013 case "$param" in
2014 # enable ftplicity preview mode
2015 '--preview')
2016 PREVIEW=1
2017 ;;
2018 # interpret duplicity disable encr switch
2019 '--disable-encryption')
2020 GPG_KEY='disabled'
2021 ;;
2022 *)
2023 if [ `echo "$param" | grep -e "^-"` ] || \
2024 [ `echo "$last_param" | grep -e "^-"` ] ; then
2025 # forward parameter[/option pairs] to duplicity
2026 dupl_opts["${#dupl_opts[@]}"]=${param}
2027 else
2028 # anything else must be a parameter (e.g. for fetch, ...)
2029 ftpl_pars["${#ftpl_pars[@]}"]=${param}
2030 fi
2031 last_param=${param}
2032 ;;
2033 esac
2034 done
2035
2036 # plausibility check config - VARS & KEY ################################################
2037 # check if src, trg, trg pw
2038 # auth info sufficient
2039 # gpg key, gpg pwd (might be empty) set in config
2040 # OR key in local gpg db
2041 # OR key can be imported from keyfile
2042 # OR fail
2043 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
2044 error " Source Path (setting SOURCE) not set or still default value in conf file
2045 '$CONF'."
2046
2047 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2048 error " Backup Target (setting TARGET) not set or still default value in conf file
2049 '$CONF'."
2050
2051 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2052 [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2053 error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}')
2054 are configured with different values. There can be only one.
2055
2056 Hint: Remove conflicting setting."
2057
2058 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2059 [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2060 error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}')
2061 are configured with different values. There can be only one.
2062
2063 Hint: Remove conflicting setting."
2064 fi
2065
2066 # GPG config plausibility check1 (disabled check) #############################
2067 if gpg_disabled; then
2068 : # encryption disabled, all is well
2069 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2070 warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file
2071 '$CONF'.
2072 Will disable encryption for duplicity now.
2073
2074 Hint:
2075 If you really want to use _no_ encryption you can disable this warning by
2076 setting GPG_KEY='disabled' in conf file."
2077 GPG_KEY='disabled'
2078 fi
2079
2080 # GPG availability check (now we know if gpg is really needed)#################
2081 if ! gpg_disabled; then
2082 gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2083 fi
2084
2085 # Output versions info ########################################################
2086 using_info
2087
2088 # GPG create key settings, config check2 (needs gpg) ##########################
2089 if gpg_disabled; then
2090 : # the following tests are not necessary
2091 else
2092
2093 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2094 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2095
2096 # enc key still default?
2097 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then
2098 error_gpg "Encryption Key GPG_KEY still default in conf file
2099 '$CONF'."
2100 fi
2101
2102 # create array of gpg encr keys, for further processing
2103 OIFS="$IFS" IFS=$'\n'
2104 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2105 IFS="$OIFS"
2106
2107 # pw set?
2108 # symmetric needs one, always
2109 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2110 ; then
2111 error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption)
2112 is empty/not set or still default value in conf file
2113 '$CONF'."
2114 fi
2115 # this is a technicality, we can only pump one pass via pipe into gpg
2116 # but symmetric already always needs one for encryption
2117 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2118 [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2119 error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2120 '$CONF'.
2121 This is unfortunately impossible. For details see duplicity manpage,
2122 section 'A Note On Symmetric Encryption And Signing'.
2123
2124 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2125 Tip2: Use gpg-agent."
2126 fi
2127
2128 # test - GPG KEY AVAILABILITY ##################################################
2129
2130 # check gpg public keys availability, try import if needed
2131 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2132 KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2133 # test availability, try to import, retest
2134 if ! gpg_pub_avail "${KEY_ID}"; then
2135 echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2136 gpg_import "${KEY_ID}" PUB
2137 gpg_key_cache RESET "${KEY_ID}"
2138 gpg_pub_avail "${KEY_ID}" || { \
2139 gpg_testing && error_gpg \
2140 "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2141 "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2142 as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2143 If not you can put it there and $ME_NAME will autoimport it on the next run.
2144 Alternatively import it manually as the user you plan to run $ME_NAME with."
2145 }
2146 else
2147 echo "Public key '${KEY_ID}' found in keychain."
2148 fi
2149 done
2150
2151 # check gpg encr secret encryption keys availability and fail
2152 # if none is available after a round of importing trials
2153 gpg_key_decryptable || \
2154 {
2155 echo "Missing secret keys for decryption in keychain."
2156 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2157 KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2158 # test availability, try to import, retest
2159 if ! gpg_sec_avail "${KEY_ID}"; then
2160 echo "Try to import secret key '${KEY_ID}' from profile."
2161 gpg_import "${KEY_ID}" SEC
2162 gpg_key_cache RESET "${KEY_ID}"
2163 fi
2164 done
2165 gpg_key_decryptable || \
2166 {
2167 gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2168 has a secret key in the keychain. Decryption will be impossible!"
2169 }
2170 }
2171
2172 # gpg secret sign key availability
2173 # if none set, autoset first encryption key as sign key
2174 if ! gpg_signing; then
2175 echo "Signing disabled per configuration."
2176 # try first key, if one set
2177 elif ! var_isset 'GPG_KEY_SIGN'; then
2178 KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2179 if [ -z "${KEY_ID}" ]; then
2180 echo "Signing disabled. No GPG_KEY entries in config."
2181 GPG_KEY_SIGN='disabled'
2182 else
2183 # use avail OR try import OR fail
2184 if gpg_sec_avail "${KEY_ID}"; then
2185 GPG_KEY_SIGN="${KEY_ID}"
2186 else
2187 echo "Signing secret key '${KEY_ID}' not found."
2188 gpg_import "${KEY_ID}" SEC
2189 gpg_key_cache RESET "${KEY_ID}"
2190 if gpg_sec_avail "${KEY_ID}"; then
2191 GPG_KEY_SIGN="${KEY_ID}"
2192 fi
2193 fi
2194
2195 # interpret sign key setting
2196 if var_isset 'GPG_KEY_SIGN'; then
2197 echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2198 else
2199 echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2200 GPG_KEY_SIGN='disabled'
2201 fi
2202 fi
2203 else
2204 KEY_ID="${GPG_KEY_SIGN}"
2205 if ! gpg_sec_avail "${KEY_ID}"; then
2206 inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2207 gpg_import "${KEY_ID}" SEC
2208 gpg_key_cache RESET "${KEY_ID}"
2209 gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2210 fi
2211 fi
2212
2213 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2214 # key enc can deal without, but might profit from gpg-agent
2215 # if GPG_PW is not set alltogether
2216 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2217 if ! gpg_symmetric && \
2218 ( ! var_isset GPG_PW || \
2219 ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2220
2221 if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2222 warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2223 elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2224 warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2225 elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2226 warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2227 else
2228 echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2229 GPG_USEAGENT="--use-agent"
2230 fi
2231 fi
2232
2233 # end GPG config plausibility check2
2234 fi
2235
2236 # config plausibility check - SPACE ###########################################
2237
2238 # is tmp is a folder and writable
2239 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2240 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2241 if [ "$?" != "0" ]; then
2242 error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2243 fi
2244
2245
2246 # get volsize, default duplicity volume size is 25MB since v0.5.07
2247 VOLSIZE=${VOLSIZE:-25}
2248 # double if asynch is on
2249 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2250
2251 # TODO: check for enough (async= upload space and WARN only
2252 # use function tmp_space
2253 #echo TODO: reimplent tmp space check
2254
2255
2256 # test - GPG SANITY #####################################################################
2257 # if encryption is disabled, skip this whole section
2258 if gpg_disabled; then
2259 echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2260 elif ! gpg_testing; then
2261 echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2262 else
2263
2264 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2265 function cleanup_gpgtest {
2266 echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2267 rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2268 }
2269
2270 # signing enabled?
2271 if gpg_signing; then
2272 CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2273 CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2274 fi
2275
2276 # using keys
2277 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2278
2279 for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2280 CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2281 done
2282 # check encrypting
2283 CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2284 run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2285 CMD_ERR=$?
2286
2287 if [ "$CMD_ERR" != "0" ]; then
2288 KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2289 [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2290 'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2291 can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2292 error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2293 fi
2294
2295 # check decrypting
2296 CMD_MSG="Test - Decrypt"
2297 gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2298 run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2299 CMD_ERR=$?
2300
2301 if [ "$CMD_ERR" != "0" ]; then
2302 error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2303 fi
2304
2305 # symmetric only
2306 else
2307 # check encrypting
2308 CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2309 run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2310 CMD_ERR=$?
2311 if [ "$CMD_ERR" != "0" ]; then
2312 error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2313 fi
2314
2315 # check decrypting
2316 CMD_MSG="Test - Decryption with passphrase"
2317 run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2318 CMD_ERR=$?
2319 if [ "$CMD_ERR" != "0" ]; then
2320 error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2321 fi
2322 fi
2323
2324 # compare original w/ decryptginal
2325 CMD_MSG="Test - Compare"
2326 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2327 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2328 CMD_ERR=$?
2329 if [ "$CMD_ERR" = "0" ]; then
2330 cleanup_gpgtest
2331 else
2332 error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2333 fi
2334
2335 fi # end disabled
2336
2337 ## an empty line
2338 #echo
2339
2340 # Exclude file is needed, create it if necessary
2341 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2342
2343 # export only used keys, if bkp not already exists ######################################
2344 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2345
2346
2347 # command execution #####################################################################
2348
2349 # urldecode url vars into plain text
2350 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2351 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2352
2353 # defined TARGET_USER&PASS vars replace their URL pendants
2354 # (double defs already dealt with)
2355 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2356 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2357
2358 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2359
2360 # issue some warnings
2361 case "$TARGET_URL_PROT_lowercase" in
2362 'cf+http')
2363 # info on missing AUTH_URL
2364 if ! var_isset 'CLOUDFILES_AUTHURL'; then
2365 inform "No CLOUDFILES_AUTHURL exported (in conf).
2366 Will use default which is probably rackspace."
2367 fi
2368 ;;
2369 'swift')
2370 # info on possibly missing AUTH_URL
2371 var_isset 'SWIFT_AUTHURL' ||\
2372 warning "\
2373 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2374 ;;
2375 'rsync')
2376 # everything in url (this backend does not support pass in env var)
2377 # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2378 # print warning older version is detected
2379 duplicity_version_lt 610 &&
2380 warning "\
2381 Duplicity version '$DUPL_VERSION' does not support providing the password as
2382 env var for rsync backend. For security reasons you should consider to
2383 update to a version greater than '0.6.10' of duplicity."
2384 ;;
2385 esac
2386
2387
2388 # for all protocols we put username in url and pass into env var
2389 # for sec�rity reasons, we url_encode username to protect special chars
2390 # first sortout backends with special ways to handle password
2391 case "$TARGET_URL_PROT_lowercase" in
2392 'imap'|'imaps')
2393 var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2394 ;;
2395 *)
2396 # rest uses FTP_PASS var
2397 var_isset 'TARGET_URL_PASS' && \
2398 BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2399 ;;
2400 esac
2401 # insert url encoded username into target url if needed
2402 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2403 BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2404 else
2405 BACKEND_URL="$TARGET"
2406 fi
2407
2408
2409 # protect eval from special chars in url (e.g. open ')' in password,
2410 # spaces in path, quotes) happens above in duplify() via quotewrap()
2411 SOURCE="$SOURCE"
2412 BACKEND_URL="$BACKEND_URL"
2413 EXCLUDE="$EXCLUDE"
2414 # since 0.7.03 --exclude-globbing-filelist is deprecated
2415 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist"
2416
2417 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2418 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2419 gsub(/\+/,\"_and_\",cmds);\
2420 gsub(/\-/,\"_or_\",cmds);\
2421 gsub(/\[/,\"_groupIn_\",cmds);\
2422 gsub(/\]/,\"_groupOut_\",cmds);\
2423 print cmds}")
2424
2425 # split commands by '_', preserve spaces even if not allowed :)
2426 IFS='_' read -ra CMDS_IN <<< "$(tolower $cmds)"
2427
2428 # convert cmds to array,
2429 # post process, translate batch commands
2430 # ATTENTION: commands are lowercase from here on out
2431 declare -a CMDS
2432 for cmd in "${CMDS_IN[@]}"; do
2433 case "$cmd" in
2434 # backup -> [pre_bkp_post]
2435 'backup')
2436 CMDS=("${CMDS[@]}" groupin pre bkp post groupout)
2437 ;;
2438 # purgeAuto -> [purge purgeFull purgeIncr] depending on set conf vars
2439 'purgeauto')
2440 purgeAuto=${MAX_AGE:+ purge}${MAX_FULL_BACKUPS:+ purgefull}${MAX_FULLS_WITH_INCRS:+ purgeincr}
2441 [[ -z "$purgeAuto" ]] && error "Command 'fullAuto' was given but neither of the purge conf vars configured."
2442 CMDS=("${CMDS[@]}" groupin $purgeAuto groupout)
2443 ;;
2444 *)
2445 CMDS=("${CMDS[@]}" "$cmd")
2446 ;;
2447 esac
2448 done
2449 #echo $(IFS=',';echo "${CMDS[*]}")
2450
2451 unset FTPL_ERR
2452
2453 # run CMDS
2454 for cmd in "${CMDS[@]}";
2455 do
2456
2457 ## init
2458 # raise index in cmd array for pre/post param
2459 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2460
2461 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2462
2463 # get next cmd,cnd vars
2464 nextno=$(( $CMD_NO ))
2465 while ! var_isset 'CMD_NEXT'
2466 do
2467 nextno=$(($nextno+1))
2468 if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2469 CMD_VALUE=${CMDS[$nextno]}
2470 is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2471 is_groupMarker "$CMD_VALUE" && continue
2472 CMD_NEXT="$CMD_VALUE"
2473 else
2474 CMD_NEXT='END'
2475 fi
2476 done
2477
2478 # get prev cnd, cnds are skipped pseudocmds
2479 prevno=$(( $CMD_NO ));
2480 while ! var_isset 'CND_PREV'
2481 do
2482 prevno=$(($prevno-1))
2483 if [ "$prevno" -ge 0 ]; then
2484 CMD_VALUE=${CMDS[$prevno]}
2485 is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2486 is_command "$CMD_VALUE" && break
2487 else
2488 break
2489 fi
2490 done
2491
2492 # get prev cmd command minus skipped commands, only executed
2493 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2494 while ! var_isset 'CMD_PREV'
2495 do
2496 prevno=$(($prevno-1))
2497 if [ "$prevno" -ge 0 ]; then
2498 CMD_VALUE=${CMDS[$prevno]}
2499 is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2500 is_groupMarker "$CMD_VALUE" && continue
2501 CMD_PREV="$CMD_VALUE"
2502 else
2503 CMD_PREV='START'
2504 fi
2505 done
2506
2507 function get_cmd_skip_count {
2508 # find closing bracket, get group skip count
2509 local nextno=$CMD_NO
2510 local GRP_OPEN=0
2511 local GRP_SKIP=0
2512 local CMD_VALUE
2513 while [ "$nextno" -lt "${#CMDS[@]}" ]
2514 do
2515 nextno=$(($nextno+1))
2516 CMD_VALUE=${CMDS[$nextno]}
2517 GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2518 if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2519 break;
2520 elif [ "$CMD_VALUE" == 'groupin' ]; then
2521 GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2522 elif [ "$CMD_VALUE" == 'groupout' ]; then
2523 GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2524 if [ "$GRP_OPEN" -lt 1 ]; then
2525 break;
2526 fi
2527 fi
2528 done
2529
2530 echo $GRP_SKIP;
2531 }
2532
2533 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2534 unset SKIP_NOW
2535 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2536 # skip cnd/grp cmds silently
2537 is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2538 CMD_SKIP=$(($CMD_SKIP - 1))
2539 SKIP_NOW="yes"
2540 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2541 CMD_SKIP=$(get_cmd_skip_count)
2542 # incl. this "cmd"
2543 CMD_SKIP=$(( $CMD_SKIP + 1 ))
2544 unset CMD_SKIPPED
2545 SKIP_NOW="yes"
2546 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2547 CMD_SKIP=$(get_cmd_skip_count)
2548 # incl. this "cmd"
2549 CMD_SKIP=$(( $CMD_SKIP + 1 ))
2550 unset CMD_SKIPPED
2551 SKIP_NOW="yes"
2552 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2553 unset 'CMD_SKIP';
2554 SKIP_NOW="yes"
2555 fi
2556
2557 # let's do the skip now
2558 if [ -n "$SKIP_NOW" ]; then
2559 # sum up how many commands we actually skipped for the prev var routines
2560 CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2561 continue
2562 fi
2563
2564 # save start time
2565 RUN_START=$(nsecs)
2566
2567 # export some useful env vars for external scripts/programs to use
2568 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2569 TARGET_URL_USER TARGET_URL_PASS \
2570 GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2571 GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2572 RUN_START
2573
2574 # user info
2575 echo; separator "Start running command $(toupper $cmd) at $(datefull_from_nsecs $RUN_START)"
2576
2577 case "$(tolower $cmd)" in
2578 'pre'|'post')
2579 if [ "$cmd" == 'pre' ]; then
2580 script=$PRE
2581 else
2582 script=$POST
2583 fi
2584 # script execution in a subshell, protect us from failures/var overwrites
2585 ( run_script "$script" )
2586 ;;
2587 'bkp')
2588 duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2589 "$SOURCE" "$BACKEND_URL"
2590 ;;
2591 'incr')
2592 duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2593 "$SOURCE" "$BACKEND_URL"
2594 ;;
2595 'full')
2596 duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2597 "$SOURCE" "$BACKEND_URL"
2598 ;;
2599 'verify')
2600 TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2601 duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2602 "$BACKEND_URL" "$SOURCE"
2603 ;;
2604 'verifypath')
2605 TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2606 IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
2607 ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2608
2609 Hint:
2610 Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2611
2612 duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2613 --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2614 ;;
2615 'list')
2616 # time param exists since 0.5.10+
2617 TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2618 duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2619 ;;
2620 'cleanup')
2621 duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2622 ;;
2623 'purge')
2624 MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2625 [ -z "$MAX_AGE" ] && error " Missing parameter <max_age>. Can be set in profile or as command line parameter."
2626
2627 duplify remove-older-than "${MAX_AGE}" \
2628 -- "${dupl_opts[@]}" "$BACKEND_URL"
2629 ;;
2630 'purgefull')
2631 MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2632 [ -z "$MAX_FULL_BACKUPS" ] && error " Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2633
2634 duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2635 -- "${dupl_opts[@]}" "$BACKEND_URL"
2636 ;;
2637 'purgeincr')
2638 MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2639 [ -z "$MAX_FULLS_WITH_INCRS" ] && error " Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2640
2641 duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2642 -- "${dupl_opts[@]}" "$BACKEND_URL"
2643 ;;
2644 'restore')
2645 OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2646 [ -z "$OUT_PATH" ] && error " Missing parameter target_path for restore.
2647
2648 Hint:
2649 Syntax is -> $ME <profile> restore <target_path> [<age>]"
2650
2651 duplify -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2652 ;;
2653 'fetch')
2654 IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
2655 TIME="${ftpl_pars[2]:-now}";
2656 ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <src_path> or <target_path> for fetch.
2657
2658 Hint:
2659 Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2660
2661 # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2662 duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2663 --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2664 ;;
2665 'status')
2666 duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2667 ;;
2668 *)
2669 error " Unknown command '$cmd'.
2670
2671 Hint:
2672 Use '$ME usage' to get usage help."
2673 ;;
2674 esac
2675
2676 CMD_ERR=$?
2677 RUN_END=$(nsecs)
2678 RUNTIME=$(( $RUN_END - $RUN_START ))
2679
2680 # print message on error; set error code
2681 if [ "$CMD_ERR" -ne 0 ]; then
2682 error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2683 FTPL_ERR=1
2684 fi
2685
2686 separator "Finished state $(error_to_string $CMD_ERR) at $(datefull_from_nsecs $RUN_END) - \
2687 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2688
2689 done
2690
2691 exit ${FTPL_ERR}
