Duply-code

From duply
Jump to: navigation, search

duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

http://ftplicity.svn.sourceforge.net/viewvc/ftplicity/duply/trunk/

SVN access

svn co https://ftplicity.svn.sourceforge.net/svnroot/ftplicity/duply/trunk duply

SVN Log

http://ftplicity.svn.sourceforge.net/viewvc/ftplicity/duply/?view=log

Latest Development Snapshot

mod time 2020-02-22

plain/text -> http://duply.net/tmp/duply.sh

  1. #!/usr/bin/env bash
  2. #
  3. ################################################################################
  4. # duply (grown out of ftplicity), is a shell front end to duplicity that #
  5. # simplifies the usage by managing settings for backup jobs in profiles. #
  6. # It supports executing multiple commands in a batch mode to enable single #
  7. # line cron entries and executes pre/post backup scripts. #
  8. # Since version 1.5.0 all duplicity backends are supported. Hence the name #
  9. # changed from ftplicity to duply. #
  10. # See http://duply.net or http://ftplicity.sourceforge.net/ for more info. #
  11. # (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany #
  12. # (c) 2008-2019 Edgar Soldin (changes since version 1.3) #
  13. ################################################################################
  14. # LICENSE: #
  15. # This program is licensed under GPLv2. #
  16. # Please read the accompanying license information in gpl-2.0.txt. #
  17. ################################################################################
  18. # TODO/IDEAS/KNOWN PROBLEMS:
  19. # - possibility to restore time frames (incl. deleted files)
  20. # realizable by listing each backup and restore from
  21. # oldest to the newest, problem: not performant
  22. # - search file in all backups function and show available
  23. # versions with backups date (list old avail since 0.6.06)
  24. # - edit profile opens conf file in vi
  25. # - implement log-fd interpretation
  26. # - add a duplicity option check against the options pending
  27. # deprecation since 0.5.10 namely --time-separator
  28. # --short-filenames
  29. # --old-filenames
  30. # - add 'exclude_<command>' list usage e.g. exclude_verify
  31. # - featreq 25: a download/install duplicity option
  32. # - hint on install software if a piece is missing
  33. # - import/export profile from/to .tgz function !!!
  34. #
  35. # CHANGELOG:
  36. # 2.2.2 ()
  37. # - bugfix 120: Failures in "Autoset trust of key" during restore
  38. # because of gpg2.2 fingerprint output change
  39. #
  40. # 2.2.1 (22.01.2020)
  41. # - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  42. # - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  43. # - fix abbreviation spelling of 'e.g.'
  44. #
  45. # 2.2 (30.12.2018)
  46. # - featreq 44: implement grouping for batch commands
  47. # new separators are [] (square brackets) or groupIn/groupOut
  48. # command 'backup' translates now to [pre_bkp_post] to be skipped as
  49. # one block in case a condition was set in the batch instruction
  50. #
  51. # 2.1 (23.07.2018)
  52. # - be more verbose when duplicity version detection fails
  53. # - using info shows python binary's path for easier identification now
  54. # - reworked python interpreter handling, it's either
  55. # configured per PYTHON var
  56. # unconfigured, parsed from duplicity shebang
  57. # or set to current duplicity default 'python2' (was 'python' until now)
  58. # - do not quotewrap strings because of slashes (e.g. paths) anymore
  59. # - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  60. #
  61. # 2.0.4 (20.02.2018)
  62. # - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  63. # - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  64. # - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  65. # - bugfix debian 882159: duply: occasionally shows negative runtimes
  66. #
  67. # 2.0.3 (29.08.2017)
  68. # - bugfix: "line 2231: CMDS: bad array subscript"
  69. # - bugfix 112: "env: illegal option -- u" on MacOSX
  70. #
  71. # 2.0.2 (23.05.2017)
  72. # - bugfix: never insert creds into file:// targets
  73. # - bugfix: avail profiles hint sometimes shortend the names by one char
  74. # - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  75. # - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  76. # - bugfix 97: Unknown command should be ERROR, not WARNING
  77. #
  78. # 2.0.1 (16.11.2016)
  79. # - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
  80. #
  81. # 2.0 (27.10.2016)
  82. # made this a major version change, as we broke backward compatibility anyway
  83. # (see last change in v1.10). got complaints that rightfully pointed out
  84. # that should only come w/ a major version change. so, here we go ;)
  85. # if your backend stops working w/ this version create a new profile and
  86. # export the env vars needed as described in the comments of the conf file
  87. # directly above the SOURCE setting.
  88. # Changes:
  89. # - making sure multi spaces in TARGET survive awk processing
  90. # - new env var PROFILE exported to scripts
  91. # - fix 102: expose a unique timestamp variable for pre/post scripts
  92. # actually a featreq. exporting RUN_START nanosec unix timestamp
  93. # - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
  94. # - fix 96: duply cannot handle two consecutive spaces in paths
  95. #
  96. # 1.11.3 (29.5.2016)
  97. # - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
  98. #
  99. # 1.11.2 (11.2.2016)
  100. # - fix "gpg: unsafe" version print out
  101. # - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
  102. # - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional
  103. # documentation about needed env vars to template conf file
  104. #
  105. # 1.11.1 (18.12.2015)
  106. # - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
  107. # - reverted duply's default PYTHON to 'python'
  108. #
  109. # 1.11 (24.11.2015)
  110. # - remove obsolete --ssh-askpass routine
  111. # - add PYTHON conf var to allow global override of used python interpreter
  112. # - enforced usage of "python2" in PATH as default interpreter for internal
  113. # use _and_ to run duplicity (setup.py changed the shebang to the fixed
  114. # path /usr/bin/python until 0.7.05, which we circumvent this way)
  115. # - featreq 36: support gpg-connect-agent as a means to detect if an agent is
  116. # running (thx Thomas Harning Jr.), used gpg-agent for detection though
  117. # - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
  118. # - key export routine respects gpg-agent usage now
  119. #
  120. # 1.10.1 (19.8.2015)
  121. # - bugfix 86: Duply+Swift outputs warning
  122. # - bugfix 87: Swift fails without BACKEND_URL
  123. #
  124. # 1.10 (31.7.2015)
  125. # - featreq 37: busybox issues - fix awk, grep version detection,
  126. # fix grep failure because --color=never switch is unsupported
  127. # (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
  128. # - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
  129. # (thx Joachim Wiedorn, also for maintaining the debian package)
  130. # - implemented base-/dirname as bash functions
  131. # - featreq 31 " Support for duplicity Azure backend " - ignored a
  132. # contributed patch by Scott McKenzie and instead opted for removing almost
  133. # all code that deals with special env vars required by backends.
  134. # adding and modifying these results in too much overhead so i dropped this
  135. # feature. the future alternative for users is to consult the duplicity
  136. # manpage and add the needed export definitions to the conf file.
  137. # appended a commented example to the template conf below the auth section.
  138. #
  139. # 1.9.2 (21.6.2015)
  140. # - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
  141. # - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
  142. # - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
  143. # - added GPG conf var (see conf template for details)
  144. # - added grep version output as it is an integral needed binary
  145. # - added PYTHONPATH printout in version output
  146. #
  147. # 1.9.1 (13.10.2014)
  148. # - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
  149. # - bugfix: CMD_PREV contained command even if it was skipped
  150. #
  151. # 1.9.0 (24.8.2014)
  152. # - bugfix: env vars were not exported when external script was executable
  153. # - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.)
  154. # see gpg manpage, section "How to specify a user ID"
  155. # let gpg complain when the delivered values are invalid for whatever reason
  156. # - started to rework tmp space checking, exposed folder & writable check
  157. # TODO: reimplement enough file space available checking
  158. #
  159. # 1.8.0 (13.7.2014)
  160. # - add command verifyPath to expose 'verify --file-to-restore' action
  161. # - add time parameter support to verify command
  162. # - add section time formats to usage output
  163. #
  164. # 1.7.4 (24.6.2014)
  165. # - remove ubuntu one support, service is discontinued
  166. # - featreq 31: add authenticated swift (contributed by Justus Seifert)
  167. #
  168. # 1.7.3 (3.4.2014)
  169. # - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
  170. #
  171. # 1.7.2 (1.4.2014 "April,April")
  172. # - bugfix: debian Bug#743190 "duply no longer allows restoration without
  173. # gpg passphrase in conf file"
  174. # GPG_AGENT_INFO env var is now needed to trigger --use-agent
  175. # - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
  176. #
  177. # 1.7.1 (30.3.2014)
  178. # - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to
  179. # incompatibility with new minus batch separator
  180. #
  181. # 1.7.0 (20.3.2014)
  182. # - disabled gpg key id plausibility check, too many valid possibilities
  183. # - featreq 7 "Halt if precondition fails":
  184. # added and(+), or(-) batch command(separator) support
  185. # - featreq 26 "pre/post script with shebang line":
  186. # if a script is flagged executable it's executed in a subshell
  187. # now as opposed to sourced to bash, which is the default
  188. # - bugfix: do not check if dpbx, swift credentials are set anymore
  189. # - bugfix: properly escape profile name, archdir if used as arguments
  190. # - add DUPL_PRECMD conf setting for use with e.g. trickle
  191. #
  192. # 1.6.0 (1.1.2014)
  193. # - support gs backend
  194. # - support dropbox backend
  195. # - add gpg-agent support to gpg test routines
  196. # - autoenable --use-agent if passwords were not defined in config
  197. # - GPG_OPTS are now honored everywhere, keyrings or complete gpg
  198. # homedir can thus be configured to be located anywhere
  199. # - always import both secret and public key if avail from config profile
  200. # - new explanatory comments in initial exclude file
  201. # - bugfix 7: Duply only imports one key at a time
  202. #
  203. # 1.5.11 (19.07.2013)
  204. # - purge-incr command for remove-all-inc-of-but-n-full feature added
  205. # patch provided by Moritz Augsburger, thanks!
  206. # - documented version command in man page
  207. #
  208. # 1.5.10 (26.03.2013)
  209. # - minor indent and documentation fixes
  210. # - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
  211. # - bugfix: fix url_decoding generally and for python3
  212. # - bugfix 3609075: wrong script results in status line (thx David Epping)
  213. #
  214. # 1.5.9 (22.11.2012)
  215. # - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
  216. # - restore/fetch now also ignores --include* or --exclude='foobar'
  217. #
  218. # 1.5.8 (26.10.2012)
  219. # - bugfix 3575487: implement proper cloud files support
  220. #
  221. # 1.5.7 (10.06.2012)
  222. # - bugfix 3531450: Cannot use space in target URL (file:///) anymore
  223. #
  224. # 1.5.6 (24.5.2012)
  225. # - commands purge, purge-full have no default value anymore for security
  226. # reasons; instead max value can be given via cmd line or must be set
  227. # in profile; else an error is shown.
  228. # - minor man page modifications
  229. #
  230. # versioning scheme will be simplified to [major].[minor].[patch] version
  231. # with the next version raise
  232. #
  233. # 1.5.5.5 (4.2.2012)
  234. # - bugfix 3479605: SEL context confused profile folder's permission check
  235. # - colon ':' in url passphrase got ignored, added python driven url_decoding
  236. # for user & pass to better process special chars
  237. #
  238. # 1.5.5.4 (16.10.2011)
  239. # - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
  240. # - add support for separate sign passphrase (needs duplicity 0.6.14+)
  241. #
  242. # 1.5.5.3 (1.10.2011)
  243. # - bugfix 3416690: preview threw echo1 error
  244. # - fix unknown cmds error usage & friends if more than 2 params were given
  245. #
  246. # 1.5.5.2 (23.9.2011)
  247. # - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
  248. # - bugfix: mawk does not support \W and did not split multikey definitions
  249. # - all parameters should survive single (') and double (") quotes now
  250. #
  251. # 1.5.5.1 (7.6.2011)
  252. # - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
  253. # - bugfix 3312208: signing detection broke symmetric gpg test routine
  254. #
  255. # 1.5.5 (2.5.2011)
  256. # - bugfix: fetch problem with space char in path, escape all params
  257. # containing non word chars
  258. # - list available profiles, if given profile cannot be found
  259. # - added --use-agent configuration hint
  260. # - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke
  261. # fetch/restore
  262. # - version command now prints out 'using installed' info
  263. # - featreq 3166169: autotrust imported keys, based on code submitted by
  264. # Martin Ellis - imported keys are now automagically trusted ultimately
  265. # - new txt2man feature to create manpages for package maintainers
  266. #
  267. # 1.5.4.2 (6.1.2011)
  268. # - new command changelog
  269. # - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
  270. # - bugfix: freebsd awk hangs on 'awk -W version'
  271. # - bugfix 3150244: mawk does not know '--version'
  272. # - minor help text improvements
  273. # - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
  274. #
  275. # 1.5.4.1 (4.12.2010)
  276. # - output awk, python, bash version now in prolog
  277. # - shebang uses /usr/bin/env now for freebsd compatibility,
  278. # bash not in /bin/bash
  279. # - new --disable-encryption parameter,
  280. # to override profile encr settings for one run
  281. # - added exclude-if-present setting to conf template
  282. # - bug 3126972: GPG_PW only needed for signing/symmetric encryption
  283. # (even though duplicity still needs it)
  284. #
  285. # 1.5.4 (15.11.2010)
  286. # - as of 1.5.3 already, new ARCH_DIR config option
  287. # - multiple key support
  288. # - ftplicity-Feature Requests-2994929: separate encryption and signing key
  289. # - key signing of symmetric encryption possible (duplicity patch committed)
  290. # - gpg tests disable switch
  291. # - gpg tests now previewable and more intelligent
  292. #
  293. # 1.5.3 (1.11.2010)
  294. # - bugfix 3056628: improve busybox compatibility, grep did not have -m param
  295. # - bugfix 2995408: allow empty password for PGP key
  296. # - bugfix 2996459: Duply erroneously escapes '-' symbol in username
  297. # - url_encode function is now pythonized
  298. # - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
  299. # - feature 3059262: Make pre and post aware of parameters,
  300. # internal parameters + CMD of pre or post
  301. #
  302. # 1.5.2.3 (16.4.2010)
  303. # - bugfix: date again, should now work virtually anywhere
  304. #
  305. # 1.5.2.2 (3.4.2010)
  306. # - minor bugfix: duplicity 0.6.8b version string now parsable
  307. # - added INSTALL.txt
  308. #
  309. # 1.5.2.1 (23.3.2010)
  310. # - bugfix: date formatting is awked now and should work on all platforms
  311. #
  312. # 1.5.2 (2.3.2010)
  313. # - bugfix: errors print to STD_ERR now, failed tasks print an error message
  314. # - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
  315. # - simplified & cleaned profileless commands, removed second instance
  316. # - generalized separator time routines
  317. # - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
  318. # - minor fixes
  319. #
  320. # 1.5.1.5 (5.2.2010)
  321. # - bugfix: added special handling of credentials for rsync, imap(s)
  322. #
  323. # 1.5.1.4 (7.1.2010)
  324. # - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
  325. # - check if ncftp binary is available if url protocol is ftp
  326. # - bugfix: duplicity output is now printed to screen directly to resolve
  327. # 'mem alloc problem' bug report
  328. # - bugfix: passwords will not be in the url anymore to solve the 'duply shows
  329. # sensitive data in process listing' bug report
  330. #
  331. # 1.5.1.3 (24.12.2009) 'merry xmas'
  332. # - bugfix: gpg pass now apostrophed to allow space and friends
  333. # - bugfix: credentials are now url encoded to allow special chars in them
  334. # a note about url encoding has been added to the conf template
  335. #
  336. # 1.5.1.2 (1.11.2009)
  337. # - bugfix: open parenthesis in password broke duplicity execution
  338. # - bugfix: ssh/scp backend does not always need credentials e.g. key auth
  339. #
  340. # 1.5.1.1 (21.09.2009)
  341. # - bugfix: fixed s3[+http] TARGET_PASS not needed routine
  342. # - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
  343. # see https://sourceforge.net/tracker/index.php?func=detail&
  344. # aid=2864410&group_id=217745&atid=1041147
  345. #
  346. # 1.5.1 (21.09.2009) - duply (fka. ftplicity)
  347. # - first things first: ftplicity (being able to support all backends since
  348. # some time) will be called duply (fka. ftplicity) from now on. The addendum
  349. # is for the time being to circumvent confusion.
  350. # - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity
  351. # command failed
  352. # - s3[+http] now supported natively by translating user/pass to access_key/
  353. # secret_key environment variables needed by duplicity s3 boto backend
  354. # - bugfix: additional output lines do not confuse version check anymore
  355. # - list command supports now age parameter (patch by stefan on feature
  356. # request tracker)
  357. # - bugfix: option/param pairs are now correctly passed on to duplicity
  358. # - bugfix: s3[+http] needs no TARGET_PASS if command is read only
  359. #
  360. # 1.5.0.2 (31.07.1009)
  361. # - bugfix: insert password in target url didn't work with debian mawk
  362. # related to previous bug report
  363. #
  364. # 1.5.0.1 (23.07.2009)
  365. # - bugfix: gawk gensub dependency raised an error on debian's default mawk
  366. # replaced with match/substr command combination (bug report)
  367. # https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
  368. # group_id=217745
  369. #
  370. # 1.5.0 (01.07.2009)
  371. # - removed ftp limitation, all duplicity backends should work now
  372. # - bugfix: date for separator failed on openwrt busybox date, added a
  373. # detecting workaround, milliseconds are not available w/ busybox date
  374. #
  375. # 1.4.2.1 (14.05.2009)
  376. # - bugfix: free temp space detection failed with lvm, fixed awk parse routine
  377. #
  378. # 1.4.2 (22.04.2009)
  379. # - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
  380. # armored ascii nature, the id helps if the key is switched for some reason
  381. # im/export routines are updated accordingly (import is backward compatible
  382. # to the old profile/gpgkey files)
  383. # - profile argument is treated as path if it contains slashes
  384. # (for details see usage)
  385. # - non-ftplicity options (all but --preview currently) are now passed
  386. # on to duplicity
  387. # - removed need for stat in secure_conf, it is ls based now
  388. # - added profile folder readable check
  389. # - added gpg version & home info output
  390. # - awk utility availability is now checked, because it was mandatory already
  391. # - tmp space is now checked on writability and space requirement
  392. # test fails on less than 25MB or configured $VOLSIZE,
  393. # test warns if there is less than two times $VOLSIZE because
  394. # that's required for --asynchronous-upload option
  395. # - gpg functionality is tested now before executing duplicity
  396. # test drive contains encryption, decryption, comparison, cleanup
  397. # this is meant to detect non trusted or other gpg errors early
  398. # - added possibility of doing symmetric encryption with duplicity
  399. # set GPG_KEY="" or simply comment it out
  400. # - added hints in config template on the depreciation of
  401. # --short-filenames, --time-separator duplicity options
  402. #
  403. # new versioning scheme 1.4.2b => 1.4.2,
  404. # beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
  405. # to the first bug fixing version and 1.4.2.2 to the second and so on
  406. # also the releases will now have a release date formatted (Day.Month.Year)
  407. #
  408. # 1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
  409. # named exactly as the profile if existing in executing dir
  410. # - improved plausibility checking of config and profile folder
  411. # - secure_conf only acts if needed and prints a warning now
  412. #
  413. # 1.4.1b - introduce status (duplicity collection-status) command
  414. # - pre/post script output printed always now, not only on errors
  415. # - new config parameter GPG_OPTS to pass gpg options
  416. # added examples & comments to profile template conf
  417. # - reworked separator times, added duration display
  418. # - added --preview switch, to preview generated command lines
  419. # - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
  420. # profiles because they have reasonable defaults now if not set
  421. #
  422. # 1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
  423. # therefore backup translates to pre_bkp_post now
  424. # - bugfix: new command bkp, which represents duplicity's
  425. # default action (incr or full if full_if_older matches
  426. # or no earlier backup chain is found)
  427. #
  428. # new versioning scheme 1.4 => 1.4.0, added new minor revision number
  429. # this is meant to slow down the rapid version growing but still keep
  430. # versions cleanly separated.
  431. # only additional features will raise the new minor revision number.
  432. # all releases start as beta, each bugfix release will raise the beta
  433. # count, usually new features arrive before a version 'ripes' to stable
  434. #
  435. # 1.4.0b
  436. # 1.4b - added startup info on version, time, selected profile
  437. # - added time output to separation lines
  438. # - introduced: command purge-full implements duplicity's
  439. # remove-all-but-n-full functionality (patch by unknown),
  440. # uses config variable $MAX_FULL_BACKUPS (default = 1)
  441. # - purge config var $MAX_AGE defaults to 1M (month) now
  442. # - command full does not execute pre/post anymore
  443. # use batch command pre_full_post if needed
  444. # - introduced batch mode cmd1_cmd2_etc
  445. # (in turn removed the bvp command)
  446. # - unknown/undefined command issues a warning/error now
  447. # - bugfix: version check works with 0.4.2 and older now
  448. # 1.3b3 - introduced pre/post commands to execute/debug scripts
  449. # - introduced bvp (backup, verify, purge)
  450. # - bugfix: removed need for awk gensub, now mawk compatible
  451. # 1.3b2 - removed pre/post need executable bit set
  452. # - profiles now under ~/.ftplicity as folders
  453. # - root can keep profiles in /etc/ftplicity, folder must be
  454. # created by hand, existing profiles must be moved there
  455. # - removed ftplicity in path requirement
  456. # - bugfix: bash < v.3 did not know '=~'
  457. # - bugfix: purge works again
  458. # 1.3 - introduces multiple profiles support
  459. # - modified some script errors/docs
  460. # - reordered gpg key check import routine
  461. # - added 'gpg key id not set' check
  462. # - added error_gpg (adds how to setup gpg key howto)
  463. # - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
  464. # - duplicity_version_check routine introduced
  465. # - added time separator, shortnames, volsize, full_if_older
  466. # duplicity options to config file (inspired by stevie
  467. # from http://weareroot.de)
  468. # 1.1.1 - bugfix: encryption reactivated
  469. # 1.1 - introduced config directory
  470. # 1.0 - first release
  471. ################################################################################
  472.  
  473. # utility functions overriding binaries
  474.  
  475. # wrap grep to override possible env set GREP_OPTIONS=--color=always
  476. function grep {
  477. command env "GREP_OPTIONS=" grep "$@"
  478. }
  479.  
  480. # implement basename in plain bash
  481. function basename {
  482. local stripped="${1%/}"
  483. echo "${stripped##*/}"
  484. }
  485.  
  486. # implement dirname in plain bash
  487. function dirname {
  488. echo ${1%/*}
  489. }
  490.  
  491. # implement basic which in plain bash
  492. function which {
  493. type -p "$@"
  494. }
  495.  
  496. # check availability of executables via file name or file paths
  497. function lookup {
  498. local bin="$1"
  499. # look for file names in path via bash hash OR
  500. # look for executables at given relative/absolute location
  501. ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
  502. }
  503.  
  504. # the python binary to use, exit code 0 when configured, else 1
  505. function python_binary {
  506. # if unset, parse from duplicity shebang
  507. if ! var_isset 'PYTHON'; then
  508. duplicity_python_binary_parse;
  509. echo $DUPL_PYTHON_BIN;
  510. return 1;
  511. else
  512. # tell if PYTHON was configured manually
  513. echo $PYTHON;
  514. return 0
  515. fi
  516. }
  517.  
  518. # important definitions #######################################################
  519.  
  520. ME_LONG="$0"
  521. ME="$(basename $0)"
  522. ME_NAME="${ME%%.*}"
  523. ME_VERSION="2.2.2(dev)"
  524. ME_WEBSITE="http://duply.net"
  525.  
  526. # default config values
  527. DEFAULT_SOURCE='/path/of/source'
  528. DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
  529. DEFAULT_TARGET_USER='_backend_username_'
  530. DEFAULT_TARGET_PASS='_backend_password_'
  531. DEFAULT_GPG='gpg'
  532. DEFAULT_GPG_KEY='_KEY_ID_'
  533. DEFAULT_GPG_PW='_GPG_PASSWORD_'
  534. DEFAULT_PYTHON='python2'
  535.  
  536. # function definitions ##########################
  537.  
  538. function set_config { # sets global config vars
  539. local CONFHOME_COMPAT="$HOME/.ftplicity"
  540. local CONFHOME="$HOME/.duply"
  541. local CONFHOME_ETC_COMPAT="/etc/ftplicity"
  542. local CONFHOME_ETC="/etc/duply"
  543.  
  544. # confdir can be delivered as path (must contain /)
  545. if [ `echo $FTPLCFG | grep /` ] ; then
  546. CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
  547. ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
  548. && echo $(pwd)/${FTPLCFG} ) || \
  549. echo ${FTPLCFG})
  550. # or DEFAULT in home/.duply folder (NEW)
  551. elif [ -d "${CONFHOME}" ]; then
  552. CONFDIR="${CONFHOME}/${FTPLCFG}"
  553. # or in home/.ftplicity folder (OLD)
  554. elif [ -d "${CONFHOME_COMPAT}" ]; then
  555. CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
  556. warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
  557. # root can put profiles under /etc/duply (NEW) if path exists
  558. elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
  559. CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
  560. # root can keep profiles under /etc/ftplicity (OLD) if path exists
  561. elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
  562. CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
  563. warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
  564. # hmm no profile folder there, then use default for error later
  565. else
  566. CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
  567. fi
  568.  
  569. # remove trailing slash, get profile name etc.
  570. CONFDIR="${CONFDIR%/}"
  571. PROFILE="${CONFDIR##*/}"
  572. CONF="$CONFDIR/conf"
  573. PRE="$CONFDIR/pre"
  574. POST="$CONFDIR/post"
  575. EXCLUDE="$CONFDIR/exclude"
  576. KEYFILE="$CONFDIR/gpgkey.asc"
  577. }
  578.  
  579. function version_info { # print version information
  580. cat <<END
  581.   $ME_NAME version $ME_VERSION
  582.   ($ME_WEBSITE)
  583. END
  584. }
  585.  
  586. function version_info_using {
  587. cat <<END
  588. $(version_info)
  589.  
  590.   $(using_info)
  591. END
  592. }
  593.  
  594. function using_info {
  595. # init needed vars into global name space
  596. lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
  597. local NOTFOUND="INVALID"
  598. local AWK_VERSION GREP_VERSION PYTHON_RUNNER
  599. # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
  600. AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
  601. GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
  602. PYTHON_RUNNER=$(python_binary)
  603. local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
  604. local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
  605. local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
  606. # print out
  607. echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
  608. ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
  609. ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
  610. ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
  611. }
  612.  
  613. function usage_info { # print usage information
  614.  
  615. cat <<USAGE_EOF
  616. VERSION:
  617. $(version_info)
  618.  
  619. DESCRIPTION:
  620. Duply deals as a wrapper for the mighty duplicity magic.
  621. It simplifies running duplicity with cron or on command line by:
  622.  
  623. - keeping recurring settings in profiles per backup job
  624. - enabling batch operations e.g. backup_verify+purge
  625. - executing pre/post scripts (different actions possible
  626. depending on previous or next command or it's exit status)
  627. - precondition checking for flawless duplicity operation
  628.  
  629. For each backup job one configuration profile must be created.
  630. The profile folder will be stored under '~/.${ME_NAME}/<profile>'
  631. (where ~ is the current users home directory).
  632. Hint:
  633. If the folder '/etc/${ME_NAME}' exists, the profiles for the super
  634. user root will be searched & created there.
  635.  
  636. USAGE:
  637. first time usage (profile creation):
  638. $ME <profile> create
  639.  
  640. general usage in single or batch mode (see EXAMPLES):
  641. $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
  642.  
  643. For batches the conditional separators can also be written as pseudo commands
  644. and(+), or(-). See SEPARATORS for details.
  645.  
  646. Non $ME options are passed on to duplicity (see OPTIONS).
  647. All conf parameters can also be defined in the environment instead.
  648.  
  649. PROFILE:
  650. Indicated by a path or a profile name (<profile>), which is resolved
  651. to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
  652.  
  653. Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
  654. the folder manually before running $ME_NAME as superuser.
  655. Note:
  656. Already existing profiles in root's home folder will cease to work
  657. unless they are moved to the new location manually.
  658.  
  659. example 1: $ME humbug backup
  660.  
  661. Alternatively a _path_ might be used e.g. useful for quick testing,
  662. restoring or exotic locations. Shell expansion should work as usual.
  663. Hint:
  664. The path must contain at least one path separator '/',
  665. e.g. './test' instead of only 'test'.
  666.  
  667. example 2: $ME ~/.${ME_NAME}/humbug backup
  668.  
  669. SEPARATORS:
  670. _ (underscore)
  671. neutral separator
  672. + (plus sign), _and_
  673. conditional AND
  674. the next command will only be executed if the previous succeeded
  675. - (minus sign), _or_
  676. conditional OR
  677. the next command will only be executed if the previous failed
  678. [] (square brackets), _groupIn_/_groupOut_
  679. enables grouping of commands
  680.  
  681. example:
  682. 'pre+[bkp-verify]_post' translates to
  683. 'pre_and_groupIn_bkp_or_verify_groupOut_post'
  684.  
  685. COMMANDS:
  686. usage get usage help text
  687.  
  688. and/or/groupIn/groupOut
  689. pseudo commands used in batches (see SEPARATORS above)
  690.  
  691. create creates a configuration profile
  692. backup backup with pre/post script execution (batch: [pre_bkp_post]),
  693. full (if full_if_older matches or no earlier backup is found)
  694. incremental (in all other cases)
  695. pre/post execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
  696. bkp as above but without executing pre/post scripts
  697. full force full backup
  698. incr force incremental backup
  699. list [<age>]
  700. list all files in backup (as it was at <age>, default: now)
  701. status prints backup sets and chains currently in repository
  702. verify [<age>] [--compare-data]
  703. list files changed, since age if given
  704. verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]
  705. list changes of a file or folder path in backup compared to a
  706. local path, since age if given
  707. restore <target_path> [<age>]
  708. restore the complete backup to <target_path> [as it was at <age>]
  709. fetch <src_path> <target_path> [<age>]
  710. fetch single file/folder from backup [as it was at <age>]
  711. purge [<max_age>] [--force]
  712. list outdated backup files (older than \$MAX_AGE)
  713. [use --force to actually delete these files]
  714. purgeFull [<max_full_backups>] [--force]
  715. list outdated backup files (\$MAX_FULL_BACKUPS being the number of
  716. full backups and associated incrementals to keep, counting in
  717. reverse chronological order)
  718. [use --force to actually delete these files]
  719. purgeIncr [<max_fulls_with_incrs>] [--force]
  720. list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being
  721. the number of full backups which associated incrementals will be
  722. kept, counting in reverse chronological order)
  723. [use --force to actually delete these files]
  724. cleanup [--force]
  725. list broken backup chain files archives (e.g. after unfinished run)
  726. [use --force to actually delete these files]
  727.  
  728. changelog print changelog / todo list
  729. txt2man feature for package maintainers - create a manpage based on the
  730. usage output. download txt2man from http://mvertes.free.fr/, put
  731. it in the PATH and run '$ME txt2man' to create a man page.
  732. version show version information of $ME_NAME and needed programs
  733.  
  734. OPTIONS:
  735. --force passed to duplicity (see commands:
  736. purge, purgeFull, purgeIncr, cleanup)
  737. --preview do nothing but print out generated duplicity command lines
  738. --disable-encryption
  739. disable encryption, overrides profile settings
  740.  
  741. TIME FORMATS:
  742. For all time related parameters like age, max_age etc.
  743. Refer to the duplicity manpage for all available formats. Here some examples:
  744. 2002-01-25T07:00:00+02:00 (full date time format string)
  745. 2002/3/5 (date string YYYY/MM/DD)
  746. 12D (interval, 12 days ago)
  747. 1h78m (interval, 1 hour 78 minutes ago)
  748.  
  749. PRE/POST SCRIPTS:
  750. Some useful internal duply variables are exported to the scripts.
  751.  
  752. PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>,
  753. GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
  754. CMD_<PREV|NEXT> (previous/next command),
  755. CND_<PREV|NEXT> (condition before/after)
  756.  
  757. The CMD_* variables were introduced to allow different actions according to
  758. the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post'
  759. will call the pre script two times, with CMD_NEXT variable set to 'bkp'
  760. on the first and to 'verify' on the second run.
  761. CMD_ERR holds the exit code of the CMD_PREV .
  762.  
  763. EXAMPLES:
  764. create profile 'humbug':
  765. $ME humbug create (don't forget to edit this new conf file)
  766. backup 'humbug' now:
  767. $ME humbug backup
  768. list available backup sets of profile 'humbug':
  769. $ME humbug status
  770. list and delete outdated backups of 'humbug':
  771. $ME humbug purge --force
  772. restore latest backup of 'humbug' to /mnt/restore:
  773. $ME humbug restore /mnt/restore
  774. restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:
  775. $ME humbug fetch etc/passwd /root/pw 4D
  776. (see "duplicity manpage", section TIME FORMATS)
  777. a one line batch job on 'humbug' for cron execution:
  778. $ME humbug backup_verify_purge --force
  779. batch job to run a full backup with pre/post scripts:
  780. $ME humbug pre_full_post
  781.  
  782. FILES:
  783. in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
  784. conf profile configuration file
  785. pre,post pre/post scripts (see above for details)
  786. gpgkey.*.asc exported GPG key files
  787. exclude a globbing list of included or excluded files/folders
  788. (see "duplicity manpage", section FILE SELECTION)
  789.  
  790. $(hint_profile)
  791.  
  792. SEE ALSO:
  793. duplicity man page:
  794. duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
  795. USAGE_EOF
  796. }
  797.  
  798. # to check call 'duply txt2man | man -l -'
  799. function usage_txt2man {
  800. usage_info | \
  801. awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
  802. txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
  803. }
  804.  
  805. function changelog {
  806. cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#( )?/,"",$0);print}'
  807. }
  808.  
  809. function create_config {
  810. if [ ! -d "$CONFDIR" ] ; then
  811. mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
  812. # create initial config file
  813. cat <<EOF >"$CONF"
  814. # gpg encryption settings, simple settings:
  815. # GPG_KEY='disabled' - disables encryption alltogether
  816. # GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
  817. # sign if secret key of key1 is available use GPG_PW for sign & decrypt
  818. # Note: you can specify keys via all methods described in gpg manpage,
  819. # section "How to specify a user ID", escape commas (,) via backslash (\)
  820. # e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
  821. # as they are used to separate the entries
  822. # GPG_PW='passphrase' - symmetric encryption using passphrase only
  823. GPG_KEY='${DEFAULT_GPG_KEY}'
  824. GPG_PW='${DEFAULT_GPG_PW}'
  825. # gpg encryption settings in detail (extended settings)
  826. # the above settings translate to the following more specific settings
  827. # GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
  828. # GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
  829. # GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
  830. # encryption. If you want to deliver different passphrases for e.g.
  831. # several keys or symmetric encryption plus key signing you can use
  832. # gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
  833. # also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage
  834. # notes on en/decryption
  835. # private key and passphrase will only be needed for decryption or signing.
  836. # decryption happens on restore and incrementals (compare archdir contents).
  837. # for security reasons it makes sense to separate the signing key from the
  838. # encryption keys. https://answers.launchpad.net/duplicity/+question/107216
  839. #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
  840. #GPG_KEY_SIGN='<prvkey>'
  841. # set if signing key passphrase differs from encryption (key) passphrase
  842. # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
  843. #GPG_PW_SIGN='<signpass>'
  844.  
  845. # uncomment and set a file path or name force duply to use this gpg executable
  846. # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
  847. #GPG='/usr/local/gpg-2.1/bin/gpg'
  848.  
  849. # gpg options passed from duplicity to gpg process (default='')
  850. # e.g. "--trust-model pgp|classic|direct|always"
  851. # or "--compress-algo=bzip2 --bzip2-compress-level=9"
  852. # or "--personal-cipher-preferences AES256,AES192,AES..."
  853. # or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
  854. # or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
  855. # also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
  856. #GPG_OPTS=''
  857.  
  858. # disable preliminary tests with the following setting
  859. #GPG_TEST='disabled'
  860.  
  861. # backend, credentials & location of the backup target (URL-Format)
  862. # generic syntax is
  863. # scheme://[user[:password]@]host[:port]/[/]path
  864. # e.g.
  865. # sftp://bob:secret@backupserver.com//home/bob/dupbkp
  866. # for details and available backends see duplicity manpage, section URL Format
  867. # http://duplicity.nongnu.org/duplicity.1.html#sect7
  868. # BE AWARE:
  869. # some backends (cloudfiles, S3 etc.) need additional env vars to be set to
  870. # work properly, read after the TARGET definition for more details.
  871. # ATTENTION:
  872. # characters other than A-Za-z0-9.-_.~ in the URL have to be
  873. # replaced by their url encoded pendants, see
  874. # http://en.wikipedia.org/wiki/Url_encoding
  875. # if you define the credentials as TARGET_USER, TARGET_PASS below $ME
  876. # will try to url_encode them for you if the need arises.
  877. TARGET='${DEFAULT_TARGET}'
  878. # optionally the username/password can be defined as extra variables
  879. # setting them here _and_ in TARGET results in an error
  880. # ATTENTION:
  881. # there are backends that do not support the user/pass auth scheme.
  882. # prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
  883. # duplicity manpage. usually there is a NOTE section explaining if and which
  884. # env vars should be set.
  885. #TARGET_USER='${DEFAULT_TARGET_USER}'
  886. #TARGET_PASS='${DEFAULT_TARGET_PASS}'
  887. # e.g. for cloud files backend it might look like this (uncomment for use!)
  888. #export CLOUDFILES_USERNAME='someuser'
  889. #export CLOUDFILES_APIKEY='somekey'
  890. #export CLOUDFILES_AUTHURL ='someurl'
  891. # the following is an incomplete list (<backend>: comma separated env vars list)
  892. # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
  893. # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
  894. # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
  895. # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
  896. # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
  897. # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
  898. # SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
  899.  
  900. # base directory to backup
  901. SOURCE='${DEFAULT_SOURCE}'
  902.  
  903. # a command that runs duplicity e.g.
  904. # shape bandwidth use via trickle
  905. # "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
  906. #DUPL_PRECMD=""
  907.  
  908. # override the used python interpreter, defaults to
  909. # - parsed result of duplicity's shebang or 'python2'
  910. # e.g. "python2" or "/usr/bin/python2.7"
  911. #PYTHON="python"
  912.  
  913. # exclude folders containing exclusion file (since duplicity 0.5.14)
  914. # Uncomment the following two lines to enable this setting.
  915. #FILENAME='.duplicity-ignore'
  916. #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
  917.  
  918. # Time frame for old backups to keep, Used for the "purge" command.
  919. # see duplicity man page, chapter TIME_FORMATS)
  920. #MAX_AGE=1M
  921.  
  922. # Number of full backups to keep. Used for the "purgeFull" command.
  923. # See duplicity man page, action "remove-all-but-n-full".
  924. #MAX_FULL_BACKUPS=1
  925.  
  926. # Number of full backups for which incrementals will be kept for.
  927. # Used for the "purgeIncr" command.
  928. # See duplicity man page, action "remove-all-inc-of-but-n-full".
  929. #MAX_FULLS_WITH_INCRS=1
  930.  
  931. # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3)
  932. # forces a full backup if last full backup reaches a specified age, for the
  933. # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
  934. # Uncomment the following two lines to enable this setting.
  935. #MAX_FULLBKP_AGE=1M
  936. #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE "
  937.  
  938. # sets duplicity --volsize option (available since v0.4.3.RC7)
  939. # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
  940. # VOLSIZE must be number of MB's to set the volume size to.
  941. # Uncomment the following two lines to enable this setting.
  942. #VOLSIZE=50
  943. #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
  944.  
  945. # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
  946. # default is 4, if not set
  947. #VERBOSITY=5
  948.  
  949. # temporary file space. at least the size of the biggest file in backup
  950. # for a successful restoration process. (default is '/tmp', if not set)
  951. #TEMP_DIR=/tmp
  952.  
  953. # Modifies archive-dir option (since 0.6.0) Defines a folder that holds
  954. # unencrypted meta data of the backup, enabling new incrementals without the
  955. # need to decrypt backend metadata first. If empty or deleted somehow, the
  956. # private key and it's password are needed.
  957. # NOTE: This is confidential data. Put it somewhere safe. It can grow quite
  958. # big over time so you might want to put it not in the home dir.
  959. # default '~/.cache/duplicity/duply_<profile>/'
  960. # if set '\${ARCH_DIR}/<profile>'
  961. #ARCH_DIR=/some/space/safe/.duply-cache
  962.  
  963. # DEPRECATED setting
  964. # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users
  965. # to change the time separator from ':' to another character that will work
  966. # on their system. HINT: For Windows SMB shares, use --time-separator='_'.
  967. # NOTE: '-' is not valid as it conflicts with date separator.
  968. # ATTENTION: only use this with duplicity < 0.5.10, since then default file
  969. # naming is compatible and this option is pending depreciation
  970. #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
  971.  
  972. # DEPRECATED setting
  973. # activates duplicity --short-filenames option, when uploading to a file
  974. # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
  975. # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
  976. # ATTENTION: only use this with duplicity < 0.5.10, later versions default file
  977. # naming is compatible and this option is pending depreciation
  978. #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
  979.  
  980. # more duplicity command line options can be added in the following way
  981. # don't forget to leave a separating space char at the end
  982. #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here "
  983.  
  984. EOF
  985.  
  986. # create initial exclude file
  987. cat <<EOF >"$EXCLUDE"
  988. # although called exclude, this file is actually a globbing file list
  989. # duplicity accepts some globbing patterns, even including ones here
  990. # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
  991. # - dir/bar/foo
  992. # + dir/bar
  993. # - **
  994. # for more details see duplicity manpage, section File Selection
  995. # http://duplicity.nongnu.org/duplicity.1.html#sect9
  996.  
  997. EOF
  998.  
  999. # Hints on first usage
  1000. cat <<EOF
  1001.  
  1002. Congratulations. You just created the profile '$FTPLCFG'.
  1003. The initial config file has been created as
  1004. '$CONF'.
  1005. You should now adjust this config file to your needs.
  1006.  
  1007. $(hint_profile)
  1008.  
  1009. EOF
  1010. fi
  1011.  
  1012. }
  1013.  
  1014. # used in usage AND create_config
  1015. function hint_profile {
  1016. cat <<EOF
  1017. IMPORTANT:
  1018.   Copy the _whole_ profile folder after the first backup to a safe place.
  1019.   It contains everything (duply related) needed to restore your backups.
  1020.  
  1021.   Pay attention to (possibly later added) external files such as credentials
  1022.   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
  1023.   (e.g. DPBX_ACCESS_TOKEN).
  1024.   It is good policy to place those in the profile folder if possible at all.
  1025.   e.g. in case of 'multi://' target the config .json file
  1026.   Env vars should be added to duply profiles' conf file.
  1027.  
  1028.   Keep access to these files restricted as they contain information (gpg key,
  1029.   passphrases etc.) to access and modify your backups.
  1030.  
  1031.   Finally:
  1032.   You should attempt a restore from an unrelated host to be sure you really
  1033.   have everything needed for restoration.
  1034.  
  1035.   Repeat these steps after _all_ configuration changes. Some configuration
  1036.   options are crucial for restoration.
  1037.  
  1038. EOF
  1039. }
  1040.  
  1041. function separator {
  1042. echo "--- $@ ---"
  1043. }
  1044.  
  1045. function inform {
  1046. echo -e "\nINFO:\n\n$@\n"
  1047. }
  1048.  
  1049. function warning {
  1050. echo -e "\nWARNING:\n\n$@\n"
  1051. }
  1052.  
  1053. function warning_oldhome {
  1054. local old=$1 new=$2
  1055. warning " ftplicity changed name to duply since you created your profiles.
  1056. Please rename the old folder
  1057. '$old'
  1058. to
  1059. '$new'
  1060. and this warning will disappear.
  1061. If you decide not to do so profiles will _only_ work from the old location."
  1062. }
  1063.  
  1064. function error_print {
  1065. echo -e "$@" >&2
  1066. }
  1067.  
  1068. function error {
  1069. error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
  1070. exit -1
  1071. }
  1072.  
  1073. function error_gpg {
  1074. [ -n "$2" ] && local hint="\n $2\n\n "
  1075.  
  1076. error "$1
  1077.  
  1078. Hint${hint:+s}:
  1079. ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
  1080. Don't forget the used _password_ as you will need it.
  1081. When done enter the 8 digit id & the password in the profile conf file.
  1082.  
  1083. The key id can be found doing a 'gpg --list-keys'. In the example output
  1084. below the key id would be FFFFFFFF for the public key.
  1085.  
  1086. pub 1024D/FFFFFFFF 2007-12-17
  1087. uid duplicity
  1088. sub 2048g/899FE27F 2007-12-17
  1089. "
  1090. }
  1091.  
  1092. function error_gpg_key {
  1093. local KEY_ID="$1"
  1094. local KIND="$2"
  1095. error_gpg "${KIND} gpg key '${KEY_ID}' cannot be found." \
  1096. "Doublecheck if the above key is listed by 'gpg --list-keys' or available
  1097. as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
  1098. If not you can put it there and $ME_NAME will autoimport it on the next run.
  1099. Alternatively import it manually as the user you plan to run $ME_NAME with."
  1100. }
  1101.  
  1102. function error_gpg_test {
  1103. [ -n "$2" ] && local hint="\n $2\n\n "
  1104.  
  1105. error "$1
  1106.  
  1107. Hint${hint:+s}:
  1108. ${hint}This error means that gpg is probably misconfigured or not working
  1109. correctly. The error message above should help to solve the problem.
  1110. However, if for some reason $ME_NAME should misinterpret the situation you
  1111. can define GPG_TEST='disabled' in the conf file to bypass the test.
  1112. Please do not forget to report the bug in order to resolve the problem
  1113. in future versions of $ME_NAME.
  1114. "
  1115. }
  1116.  
  1117. function error_path {
  1118. error "$@
  1119. PATH='$PATH'
  1120. "
  1121. }
  1122.  
  1123. function error_to_string {
  1124. [ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
  1125. }
  1126.  
  1127. function duplicity_version_get {
  1128. # nothing to do, just print
  1129. var_isset DUPL_VERSION && return
  1130.  
  1131. local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
  1132. # only run with a user specific python if configured (running by default
  1133. # breaks homebrew as they place a shell wrapper for duplicity in path)
  1134. PYTHON_BIN="$(python_binary)" &&\
  1135. CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
  1136. CMD="$CMD --version 2>&1"
  1137. DUPL_VERSION_OUT=$(eval "$CMD")
  1138. DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
  1139. #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
  1140. DUPL_VERSION_VALUE=0
  1141. DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
  1142. if (match(v,/[^\.0-9]+[0-9]*$/)){
  1143. rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
  1144. if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
  1145. split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
  1146. print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
  1147. print "DUPL_VERSION_VALUE=\047"vvalue"\047"
  1148. print "DUPL_VERSION_RC=\047"rc"\047"
  1149. print "DUPL_VERSION_SUFFIX=\047"rest"\047"
  1150. }')
  1151. eval "$DUPL_VERSION_AWK"
  1152. #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
  1153.  
  1154. # doublecheck findings and report error
  1155. if [ $DUPL_VERSION_VALUE -eq 0 ]; then
  1156. inform "duplicity version check failed (please report, this is a bug)
  1157. the command
  1158. $CMD
  1159. resulted in
  1160. $DUPL_VERSION_OUT
  1161. "
  1162. elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
  1163. error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
  1164. You should upgrade your version of duplicity to at least v0.4.4RC4 or
  1165. use the older ftplicity version 1.1.1 from $ME_WEBSITE."
  1166. fi
  1167. }
  1168.  
  1169. function duplicity_version_ge {
  1170. [ "$DUPL_VERSION_VALUE" -ge "$1" ]
  1171. }
  1172.  
  1173. function duplicity_version_lt {
  1174.  ! duplicity_version_ge "$1"
  1175. }
  1176.  
  1177. # parse interpreter from duplicity shebang
  1178. function duplicity_python_binary_parse {
  1179. # cached result
  1180. ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
  1181.  
  1182. # parse it or warn
  1183. local DUPL_BIN=$(which duplicity)
  1184. DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
  1185. if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
  1186. warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
  1187. Will assume it is '$DEFAULT_PYTHON'."
  1188. DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
  1189. fi
  1190. }
  1191.  
  1192. function run_script { # run pre/post scripts
  1193. local ERR=0
  1194. local SCRIPT="$1"
  1195. if [ ! -z "$PREVIEW" ] ; then
  1196. echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
  1197. elif [ -r "$SCRIPT" ] ; then
  1198. echo -n "Running '$SCRIPT' "
  1199. if [ -x "$SCRIPT" ]; then
  1200. OUT=$("$SCRIPT" 2>&1)
  1201. ERR=$?
  1202. else
  1203. OUT=$(. "$SCRIPT" 2>&1)
  1204. ERR=$?
  1205. fi
  1206. [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
  1207. echo -en ${OUT:+"Output: $OUT\n"} ;
  1208. else
  1209. echo "Skipping n/a script '$SCRIPT'."
  1210. fi
  1211. return $ERR
  1212. }
  1213.  
  1214. function run_cmd {
  1215. # run or print escaped cmd string
  1216. local CMD_ERR=0
  1217. if [ -n "$PREVIEW" ]; then
  1218. CMD_OUT=$( echo "$@ 2>&1" )
  1219. CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
  1220. elif [ -n "$CMD_DISABLED" ]; then
  1221. CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
  1222. else
  1223. echo -n -e "$CMD_MSG"
  1224. CMD_OUT=` eval "$@" 2>&1 `
  1225. CMD_ERR=$?
  1226. if [ "$CMD_ERR" = "0" ]; then
  1227. CMD_MSG=" (OK)"
  1228. else
  1229. CMD_MSG=" (FAILED)"
  1230. fi
  1231. fi
  1232. echo -e "$CMD_MSG"
  1233. # reset
  1234. unset CMD_DISABLED CMD_MSG
  1235. return $CMD_ERR
  1236. }
  1237.  
  1238. function qw { quotewrap "$@"; }
  1239.  
  1240. function quotewrap {
  1241. local param="$@"
  1242. # quote strings having non word chars (e.g. spaces)
  1243. if echo "$param" | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
  1244. echo "$param" | awk '{\
  1245. gsub(/[\047]/,"\047\\\047\047",$0);\
  1246. gsub(/[\042]/,"\047\\\042\047",$0);\
  1247. print "\047"$0"\047"}'
  1248. return
  1249. fi
  1250. echo $param
  1251. }
  1252.  
  1253. function duplicity_params_global {
  1254. # already done? return
  1255. var_isset 'DUPL_PARAMS_GLOBAL' && return
  1256. local DUPL_ARG_ENC
  1257.  
  1258. # use key only if set in config, else leave it to symmetric encryption
  1259. if gpg_disabled; then
  1260. local DUPL_PARAM_ENC='--no-encryption'
  1261. else
  1262. local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
  1263. gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
  1264. # interpret password settings
  1265. var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
  1266. var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
  1267. fi
  1268.  
  1269. local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
  1270.  
  1271. # set name for dupl archive folder, since 0.6.0
  1272. if duplicity_version_ge 601; then
  1273. local DUPL_ARCHDIR=''
  1274. if var_isset 'ARCH_DIR'; then
  1275. DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
  1276. # reuse erronously duply_ prefixed folders from bug #117
  1277. if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
  1278. DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
  1279. else
  1280. DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
  1281. fi
  1282. else
  1283. DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
  1284. fi
  1285. fi
  1286.  
  1287. DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
  1288. ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
  1289. ${GPG_OPTS}"
  1290.  
  1291. DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
  1292. ${DUPL_ARG_ENC}"
  1293. }
  1294.  
  1295. # function to filter the DUPL_PARAMS var from user conf
  1296. function duplicity_params_conf {
  1297. # reuse cmd var from main loop
  1298. ## in/exclude parameters are currently not supported on restores
  1299. if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
  1300. # filter exclude params from fetch/restore/status
  1301. eval "stripXcludes $DUPL_PARAMS"
  1302. return
  1303. fi
  1304.  
  1305. # nothing done, print unchanged
  1306. echo "$DUPL_PARAMS"
  1307. }
  1308.  
  1309. # strip in/exclude parameters from param string
  1310. function stripXcludes {
  1311. local STRIPNEXT OUT;
  1312. for p in "$@"; do
  1313. if [ -n "$STRIPNEXT" ]; then
  1314. unset STRIPNEXT
  1315. # strip the value of previous parameter
  1316. continue
  1317. elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
  1318. # strips e.g. --include /foo/bar
  1319. STRIPNEXT="yes"
  1320. continue
  1321. elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
  1322. # strips e.g. --include=/foo/bar
  1323. continue
  1324. fi
  1325.  
  1326. OUT="$OUT $(qw "$p")"
  1327. done
  1328. echo "$OUT"
  1329. }
  1330.  
  1331. function duplify { # the actual wrapper function
  1332. local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
  1333.  
  1334. # put command (with params) first in duplicity parameters
  1335. for param in "$@" ; do
  1336. # split cmd from params (everything before splitchar --)
  1337. if [ "$param" == "--" ] ; then
  1338. PARAMSNOW=1
  1339. else
  1340. # wrap in quotes to protect from spaces
  1341. [ ! $PARAMSNOW ] && \
  1342. DUPL_CMD="$DUPL_CMD $(qw "$param")" \
  1343. || \
  1344. DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
  1345. fi
  1346. done
  1347.  
  1348. # init global duplicity parameters same for all tasks
  1349. duplicity_params_global
  1350.  
  1351. local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
  1352. # run in cmd line preview mode if requested
  1353. var_isset 'PREVIEW' && RUN=echo
  1354. # try to resolve duplicity path for usage with python interpreter
  1355. DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
  1356. # only run with a user specific python if configured (running by default
  1357. # breaks homebrew as they place a shell wrapper for duplicity in path)
  1358. PYTHON_BIN="$(python_binary)" &&\
  1359. BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
  1360.  
  1361. $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
  1362. ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
  1363. $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
  1364.  
  1365. local ERR=$?
  1366. return $ERR
  1367. }
  1368.  
  1369. function secureconf { # secure the configuration dir
  1370. #PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
  1371. local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
  1372. if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
  1373. chmod u+rwX,go= "$CONFDIR"; local ERR=$?
  1374. warning "The profile's folder
  1375. '$CONFDIR'
  1376. permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
  1377. fi
  1378. }
  1379.  
  1380. # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
  1381. function date_fix {
  1382. local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
  1383. local date
  1384. #[ "$1" == "%N" ] && return #test the no nsec test below
  1385. # gnu date with -d @epoch
  1386. date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
  1387. echo $date && return
  1388. # date bsd,osx with -r epoch
  1389. date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
  1390. echo $date && return
  1391. # date busybox with -d epoch -D %s
  1392. date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
  1393. echo $date && return
  1394. ## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
  1395. # python fallback
  1396. date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
  1397. echo $date && return
  1398. # awk fallback
  1399. date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
  1400. echo $date && return
  1401. # perl fallback
  1402. date=$(perl -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
  1403. echo $date && return
  1404. # error
  1405. echo "ERROR"
  1406. return 1
  1407. }
  1408.  
  1409. function nsecs {
  1410. local NSECS
  1411. # test if date supports nanosecond output
  1412. if ! var_isset NSECS_DISABLED; then
  1413. NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
  1414. [ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
  1415. fi
  1416.  
  1417. # add 9 digits, not all date(s) deliver nsecs e.g. busybox date
  1418. if [ "$NSECS_DISABLED" == "1" ]; then
  1419. date_fix %s000000000
  1420. else
  1421. date_fix %s%N
  1422. fi
  1423. }
  1424.  
  1425. function nsecs_to_sec {
  1426. echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
  1427. }
  1428.  
  1429. function datefull_from_nsecs {
  1430. date_from_nsecs $1 '%F %T'
  1431. }
  1432.  
  1433. function date_from_nsecs {
  1434. local FORMAT=${2:-%T}
  1435. local TIME=$(nsecs_to_sec $1)
  1436. local SECS=${TIME%.*}
  1437. local DATE=$(date_fix "%T" ${SECS:-0})
  1438. echo $DATE.${TIME#*.}
  1439. }
  1440.  
  1441. function var_isset {
  1442. if [ -z "$1" ]; then
  1443. echo "ERROR: function var_isset needs a string as parameter"
  1444. elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
  1445. return 0
  1446. fi
  1447. return 1
  1448. }
  1449.  
  1450. function is_condition {
  1451. local CMD=$(tolower "$1")
  1452. [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
  1453. }
  1454.  
  1455. function is_groupMarker {
  1456. local CMD=$(tolower "$1")
  1457. [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
  1458. }
  1459.  
  1460. function is_command {
  1461. local CMD=$(tolower "$1")
  1462. ! is_condition "$CMD" && ! is_groupMarker "$CMD"
  1463. }
  1464.  
  1465. function url_encode {
  1466. # utilize python, silently do nothing on error - because no python no duplicity
  1467. OUT=$("$(python_binary)" -c "
  1468. try: import urllib.request as urllib
  1469. except ImportError: import urllib
  1470. print(urllib.${2}quote('$1'));
  1471. " 2>/dev/null ); ERR=$?
  1472. [ "$ERR" -eq 0 ] && echo $OUT || echo $1
  1473. }
  1474.  
  1475. function url_decode {
  1476. # reuse function above with a simple string param hack
  1477. url_encode "$1" "un"
  1478. }
  1479.  
  1480. function toupper {
  1481. echo "$@"|awk '$0=toupper($0)'
  1482. }
  1483.  
  1484. function tolower {
  1485. echo "$@"|awk '$0=tolower($0)'
  1486. }
  1487.  
  1488. function isnumber {
  1489. case "$*" in
  1490. ''|*[!0-9]*) return 1;;
  1491. *) return 0;;
  1492. esac
  1493. }
  1494.  
  1495. #function tmp_space {
  1496. #
  1497. # if ! isnumber $VOLSIZE; then
  1498. # inform "failed to determine free space (please report, this is a bug)"
  1499. # return
  1500. # fi
  1501. #
  1502. # get free temp space
  1503. # TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
  1504. # # check for free space or FAIL
  1505. # if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
  1506. # error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
  1507. #than one duplicity volume (${VOLSIZE}MB).
  1508. #
  1509. # Hint: Free space or change TEMP_DIR setting."
  1510. #fi
  1511. #
  1512. #}
  1513.  
  1514. function gpg_disabled {
  1515. echo "${GPG_KEY}" | grep -iq -e '^disabled$'
  1516. }
  1517.  
  1518. # usage: join SEPARATOR "entry1" "entry2"
  1519. function join {
  1520. local SEP="$1" ENTRY OUT; shift;
  1521. for ENTRY in "$@"; do
  1522. ENTRY=${ENTRY//$SEP/\\$SEP}
  1523. [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
  1524. done
  1525. echo $OUT
  1526. }
  1527.  
  1528. function gpg_signing {
  1529. echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
  1530. }
  1531.  
  1532. # parameter key id, key_type
  1533. function gpg_keyfile {
  1534. local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
  1535. local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
  1536. echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
  1537. }
  1538.  
  1539. # parameter key id
  1540. function gpg_import {
  1541. local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
  1542. # create a list of legacy key file names and current naming scheme
  1543. # we always import pub and sec if they are avail in conf folder
  1544. local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
  1545. $(gpg_keyfile "$KEY_ID" PUB) $(gpg_keyfile "$KEY_ID" SEC))
  1546.  
  1547. # Try autoimport from existing old gpgkey files
  1548. # and new gpgkey.XXX.asc files (since v1.4.2)
  1549. # and even newer gpgkey.XXX.[pub|sec].asc
  1550. for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
  1551. FILE=${KEYFILES[$i]}
  1552. if [ -f "$FILE" ]; then
  1553. FOUND=1
  1554.  
  1555. CMD_MSG="Import keyfile '$FILE' to keyring"
  1556. run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
  1557. if [ "$?" != "0" ]; then
  1558. warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
  1559. ERR=1
  1560. # continue with next
  1561. continue
  1562. fi
  1563. fi
  1564. done
  1565.  
  1566. if [ "$FOUND" -eq 0 ]; then
  1567. warning "No keyfile for '$KEY_ID' found in profile\n'$CONFDIR'."
  1568. fi
  1569.  
  1570. # try to set trust automagically
  1571. CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
  1572. run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
  1573. if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then
  1574. # success on all levels, we're done
  1575. return $ERR
  1576. fi
  1577.  
  1578. # failover: user has to set trust manually
  1579. echo -e "For $ME_NAME to work you have to set the trust level
  1580. with the command \"trust\" to \"ultimate\" (5) now.
  1581. Exit the edit mode of gpg with \"quit\"."
  1582. CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
  1583. run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
  1584.  
  1585. return $ERR
  1586. }
  1587.  
  1588. # see 'How to specify a user ID' on gpg manpage
  1589. function gpg_fingerprint {
  1590. gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
  1591. awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
  1592. }
  1593.  
  1594. function gpg_export_if_needed {
  1595. local SUCCESS FILE KEY_TYPE
  1596. local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
  1597. for KEY_ID in "$@"; do
  1598. # check if already exported, do it if not
  1599. for KEY_TYPE in PUB SEC; do
  1600. FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
  1601. if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
  1602. # exporting
  1603. CMD_MSG="Backup $KEY_TYPE key '$KEY_ID' to profile."
  1604. # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
  1605. # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
  1606. run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
  1607. CMD_ERR=$?
  1608.  
  1609. if [ "$CMD_ERR" = "0" ]; then
  1610. CMD_MSG="Write file '"$(basename "$FILE")"'"
  1611. run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
  1612. fi
  1613.  
  1614. if [ "$CMD_ERR" != "0" ]; then
  1615. warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
  1616. else
  1617. SUCCESS=1
  1618. fi
  1619.  
  1620. # cleanup
  1621. rm $(qw "$TMPFILE") 1>/dev/null 2>&1
  1622. fi
  1623. done
  1624. done
  1625.  
  1626. [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
  1627. You should backup your changed profile folder now and store it in a safe place."
  1628. }
  1629.  
  1630. # replace all non-alnum chars with underscore (for file operations)
  1631. function gpg_key_legalize {
  1632. echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
  1633. }
  1634.  
  1635. function gpg_key_cache {
  1636. local RES
  1637. local MODE=$1
  1638. shift
  1639. local PREFIX="GPG_KEY"
  1640. local SUFFIX=$(gpg_key_legalize "$@")
  1641. local KEYID="$*"
  1642. local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
  1643. if [ "$MODE" = "RESET" ]; then
  1644. eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
  1645. return 255
  1646. elif ! var_isset "$CACHE"; then
  1647. if [ "$MODE" = "PUB" ]; then
  1648. RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
  1649. elif [ "$MODE" = "SEC" ]; then
  1650. RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
  1651. else
  1652. return 255
  1653. fi
  1654. eval $CACHE=$RES
  1655. fi
  1656. eval return \$$CACHE
  1657. }
  1658.  
  1659. function gpg_pub_avail {
  1660. gpg_key_cache PUB "$@"
  1661. }
  1662.  
  1663. function gpg_sec_avail {
  1664. gpg_key_cache SEC "$@"
  1665. }
  1666.  
  1667. function gpg_key_format {
  1668. echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
  1669. }
  1670.  
  1671. # splits a comma separated line into lines, respects escaped commas
  1672. function gpg_split_keyset {
  1673. local LIST
  1674. LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
  1675. echo -e "$LIST"
  1676. }
  1677.  
  1678. function gpg_prefix_keyset {
  1679. local PREFIX="$1" OUT=""
  1680. shift
  1681. for KEY_ID in "$@"; do
  1682. OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
  1683. done
  1684. echo $OUT
  1685. }
  1686.  
  1687. # grep a variable from conf text file (currently not used)
  1688. function gpg_passwd {
  1689. [ -r "$CONF" ] && \
  1690. awk '/^[ \t]*GPG_PW[ \t=]/{\
  1691. sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
  1692. gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
  1693. print $0; exit}' "$CONF"
  1694. }
  1695.  
  1696. # return success if at least one secret key is available
  1697. function gpg_key_decryptable {
  1698. local KEY_ID
  1699. for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
  1700. gpg_sec_avail "$KEY_ID" && return 0
  1701. done
  1702. return 1
  1703. }
  1704.  
  1705. function gpg_symmetric {
  1706. [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
  1707. }
  1708.  
  1709. # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
  1710. function gpg_param_passwd {
  1711. var_isset GPG_USEAGENT && exit 1
  1712.  
  1713. if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
  1714. echo "--passphrase-fd 0 --batch"
  1715. fi
  1716. }
  1717.  
  1718. # select the earlist defined and create an "echo <value> |" string
  1719. function gpg_pass_pipein {
  1720. var_isset GPG_USEAGENT && exit 1
  1721.  
  1722. for var in "$@"
  1723. do
  1724. if var_isset "$var"; then
  1725. echo "echo $(qw $(eval echo \$$var)) |"
  1726. return 0
  1727. fi
  1728. done
  1729.  
  1730. return 1
  1731. }
  1732.  
  1733. # checks if gpg-agent is available, returns error code
  1734. # 0 on success
  1735. # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
  1736. # 2 if GPG_AGENT_INFO is stale
  1737. # 3 cannot connect to gpg-agent
  1738. function gpg_agent_avail {
  1739. # GPG_AGENT_INFO is deprecated in gpg2.1,
  1740. # first try to connect to a possibly running agent here
  1741. local ERR=3
  1742. gpg-agent > /dev/null 2>&1 && return 0
  1743.  
  1744. # detect stale pid in legacy GPG_AGENT_INFO env var
  1745. if var_isset GPG_AGENT_INFO; then
  1746. # check if a pid matching process is running at all
  1747. local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
  1748. if isnumber "$GPG_AGENT_PID"; then
  1749. ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
  1750. fi
  1751. fi
  1752.  
  1753. return $ERR
  1754. }
  1755.  
  1756. function gpg_custom_binary {
  1757. var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
  1758. echo "--gpg-binary $(qw "$GPG")"
  1759. }
  1760.  
  1761. function gpg_binary {
  1762. local BIN
  1763. var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
  1764. echo "$BIN"
  1765. }
  1766.  
  1767. function gpg_avail {
  1768. lookup "$(gpg_binary)"
  1769. }
  1770.  
  1771. # enforce the use of our selected gpg binary
  1772. function gpg {
  1773. command "$(gpg_binary)" "$@"
  1774. }
  1775. export -f gpg
  1776.  
  1777. # start of script #######################################################################
  1778.  
  1779. # confidentiality first, all we create is only readable by us
  1780. umask 077
  1781.  
  1782. # check if ftplicity is there & executable
  1783. [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
  1784.  
  1785. if [ ${#@} -eq 1 ]; then
  1786. cmd="${1}"
  1787. else
  1788. FTPLCFG="${1}" ; cmd="${2}"
  1789. fi
  1790.  
  1791. # deal with command before profile validation calls
  1792. # show requested version
  1793. # OR requested usage info
  1794. # OR create a profile
  1795. # OR fall through
  1796. ##if [ ${#@} -le 2 ]; then
  1797. case "$cmd" in
  1798. changelog)
  1799. changelog
  1800. exit 0
  1801. ;;
  1802. create)
  1803. set_config
  1804. if [ -d "$CONFDIR" ]; then
  1805. error "The profile '$FTPLCFG' already exists in
  1806. '$CONFDIR'.
  1807.  
  1808. Hint:
  1809. If you _really_ want to create a new profile by this name you will
  1810. have to manually delete the existing profile folder first."
  1811. exit 1
  1812. else
  1813. create_config
  1814. exit 0
  1815. fi
  1816. ;;
  1817. txt2man)
  1818. set_config
  1819. usage_txt2man
  1820. exit 0
  1821. ;;
  1822. usage|-help|--help|-h|-H)
  1823. set_config
  1824. usage_info
  1825. exit 0
  1826. ;;
  1827. version|-version|--version|-v|-V)
  1828. # profile can override GPG/PYTHON, so import it if it was given
  1829. var_isset FTPLCFG && {
  1830. set_config
  1831. [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
  1832. }
  1833. version_info_using
  1834. exit 0
  1835. ;;
  1836. # fallthrough.. we got a command that needs an existing profile
  1837. *)
  1838. # if we reach here, user either forgot profile or chose wrong profileless command
  1839. if [ ${#@} -le 1 ]; then
  1840. error "\
  1841. Missing or wrong parameters.
  1842. Only the commands
  1843. changelog, create, usage, txt2man, version
  1844. can be called without selecting an existing profile first.
  1845. Your command was '$cmd'.
  1846.  
  1847. Hint: Run '$ME usage' to get help."
  1848. fi
  1849. esac
  1850.  
  1851.  
  1852. # Hello world
  1853. echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
  1854.  
  1855. # check system environment
  1856.  
  1857. # is duplicity avail
  1858. lookup duplicity || error_path "duplicity missing. installed und available in path?"
  1859. # init, exec duplicity version check info
  1860. duplicity_python_binary_parse
  1861. duplicity_version_get
  1862.  
  1863. # check for certain important helper programs
  1864. for f in awk grep "$(python_binary)"; do
  1865. lookup "$f" || \
  1866. error_path "$f missing. installed und available in path?"
  1867. done
  1868.  
  1869. ### read configuration
  1870. set_config
  1871. # check validity
  1872. if [ ! -d "$CONFDIR" ]; then
  1873. error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
  1874. '$CONFDIR'.
  1875.  
  1876. Hints:
  1877. Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
  1878. Use '$ME <name> create' to create a new profile.
  1879. Use '$ME usage' to get usage help."
  1880. elif [ ! -x "$CONFDIR" ]; then
  1881. error "\
  1882. Profile folder in '$CONFDIR' cannot be accessed.
  1883.  
  1884. Hint:
  1885. Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
  1886. elif [ ! -f "$CONF" ] ; then
  1887. error "'$CONF' not found."
  1888. elif [ ! -r "$CONF" ] ; then
  1889. error "'$CONF' not readable."
  1890. else
  1891. . "$CONF"
  1892. #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
  1893. TEMP_DIR=${TEMP_DIR:-'/tmp'}
  1894. # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
  1895. if var_isset 'TARGET_PW'; then
  1896. TARGET_PASS="${TARGET_PW}"
  1897. fi
  1898. fi
  1899. echo "Using profile '$CONFDIR'."
  1900.  
  1901. # secure config dir, if needed w/ warning
  1902. secureconf
  1903.  
  1904. # split TARGET in handy variables
  1905. TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
  1906. target=$0; match(target,/^([^\/:]+):\/\//); \
  1907. prot=substr(target,RSTART,RLENGTH);\
  1908. rest=substr(target,RSTART+RLENGTH); \
  1909. if (credsavail=match(rest,/^[^@]*@/)){\
  1910. creds=substr(rest,RSTART,RLENGTH-1);\
  1911. credcount=split(creds,cred,":");\
  1912. rest=substr(rest,RLENGTH+1);\
  1913. # split creds with regexp\
  1914. match(creds,/^([^:]+)/);\
  1915. user=substr(creds,RSTART,RLENGTH);\
  1916. pass=substr(creds,RSTART+1+RLENGTH);\
  1917. };\
  1918. # filter quotes or escape them\
  1919. gsub(/[\047\042]/,"",prot);\
  1920. gsub(/[\047\042]/,"",rest);\
  1921. gsub(/[\047]/,"\047\\\047\047",creds);\
  1922. print "TARGET_URL_PROT=\047"prot"\047\n"\
  1923. "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
  1924. "TARGET_URL_CREDS=\047"creds"\047\n";\
  1925. if(user){\
  1926. gsub(/[\047]/,"\047\\\047\047",user);\
  1927. print "TARGET_URL_USER=\047"user"\047\n"}\
  1928. if(pass){\
  1929. gsub(/[\047]/,"\047\\\047\047",pass);\
  1930. print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
  1931. }')
  1932. eval "${TARGET_SPLIT_URL}"
  1933.  
  1934. # fetch commmand from parameters ########################################################
  1935. # Hint: cmds is also used to check if authentification info sufficient in the next step
  1936. cmds="$2"; shift 2
  1937.  
  1938. # complain if command(s) missing
  1939. [ -z $cmds ] && error " No command given.
  1940.  
  1941. Hint:
  1942. Use '$ME usage' to get usage help."
  1943.  
  1944. # process params
  1945. for param in "$@"; do
  1946. #echo !$param!
  1947. case "$param" in
  1948. # enable ftplicity preview mode
  1949. '--preview')
  1950. PREVIEW=1
  1951.  ;;
  1952. # interpret duplicity disable encr switch
  1953. '--disable-encryption')
  1954. GPG_KEY='disabled'
  1955.  ;;
  1956. *)
  1957. if [ `echo "$param" | grep -e "^-"` ] || \
  1958. [ `echo "$last_param" | grep -e "^-"` ] ; then
  1959. # forward parameter[/option pairs] to duplicity
  1960. dupl_opts["${#dupl_opts[@]}"]=${param}
  1961. else
  1962. # anything else must be a parameter (e.g. for fetch, ...)
  1963. ftpl_pars["${#ftpl_pars[@]}"]=${param}
  1964. fi
  1965. last_param=${param}
  1966.  ;;
  1967. esac
  1968. done
  1969.  
  1970. # plausibility check config - VARS & KEY ################################################
  1971. # check if src, trg, trg pw
  1972. # auth info sufficient
  1973. # gpg key, gpg pwd (might be empty) set in config
  1974. # OR key in local gpg db
  1975. # OR key can be imported from keyfile
  1976. # OR fail
  1977. if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
  1978. error " Source Path (setting SOURCE) not set or still default value in conf file
  1979. '$CONF'."
  1980.  
  1981. elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
  1982. error " Backup Target (setting TARGET) not set or still default value in conf file
  1983. '$CONF'."
  1984.  
  1985. elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
  1986. [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
  1987. error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}')
  1988. are configured with different values. There can be only one.
  1989.  
  1990. Hint: Remove conflicting setting."
  1991.  
  1992. elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
  1993. [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
  1994. error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}')
  1995. are configured with different values. There can be only one.
  1996.  
  1997. Hint: Remove conflicting setting."
  1998. fi
  1999.  
  2000. # GPG config plausibility check1 (disabled check) #############################
  2001. if gpg_disabled; then
  2002.  : # encryption disabled, all is well
  2003. elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
  2004. warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file
  2005. '$CONF'.
  2006. Will disable encryption for duplicity now.
  2007.  
  2008. Hint:
  2009. If you really want to use _no_ encryption you can disable this warning by
  2010. setting GPG_KEY='disabled' in conf file."
  2011. GPG_KEY='disabled'
  2012. fi
  2013.  
  2014. # GPG availability check (now we know if gpg is really needed)#################
  2015. if ! gpg_disabled; then
  2016. gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
  2017. fi
  2018.  
  2019. # Output versions info ########################################################
  2020. using_info
  2021.  
  2022. # GPG create key settings, config check2 (needs gpg) ##########################
  2023. if gpg_disabled; then
  2024.  : # the following tests are not necessary
  2025. else
  2026.  
  2027. # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
  2028. GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
  2029.  
  2030. # enc key still default?
  2031. if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then
  2032. error_gpg "Encryption Key GPG_KEY still default in conf file
  2033. '$CONF'."
  2034. fi
  2035.  
  2036. # create array of gpg encr keys, for further processing
  2037. OIFS="$IFS" IFS=$'\n'
  2038. GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
  2039. IFS="$OIFS"
  2040.  
  2041. # pw set?
  2042. # symmetric needs one, always
  2043. if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
  2044.  ; then
  2045. error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption)
  2046. is empty/not set or still default value in conf file
  2047. '$CONF'."
  2048. fi
  2049. # this is a technicality, we can only pump one pass via pipe into gpg
  2050. # but symmetric already always needs one for encryption
  2051. if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
  2052. [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
  2053. error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
  2054. '$CONF'.
  2055. This is unfortunately impossible. For details see duplicity manpage,
  2056. section 'A Note On Symmetric Encryption And Signing'.
  2057.  
  2058. Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
  2059. Tip2: Use gpg-agent."
  2060. fi
  2061.  
  2062. # check gpg encr public keys availability
  2063. for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
  2064. KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
  2065. # test availability, try to import, retest
  2066. if ! gpg_pub_avail "${KEY_ID}"; then
  2067. echo "Encryption public key '${KEY_ID}' not found."
  2068. gpg_import "${KEY_ID}" PUB
  2069. gpg_key_cache RESET "${KEY_ID}"
  2070. gpg_pub_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Public"
  2071. fi
  2072. done
  2073.  
  2074. # gpg secret sign key availability
  2075. # if none set, autoset first encryption key as sign key
  2076. if ! gpg_signing; then
  2077. echo "Signing disabled per configuration."
  2078. # try first key, if one set
  2079. elif ! var_isset 'GPG_KEY_SIGN'; then
  2080. KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
  2081. if [ -z "${KEY_ID}" ]; then
  2082. echo "Signing disabled. No GPG_KEY entries in config."
  2083. GPG_KEY_SIGN='disabled'
  2084. else
  2085. # use avail OR try import OR fail
  2086. if gpg_sec_avail "${KEY_ID}"; then
  2087. GPG_KEY_SIGN="${KEY_ID}"
  2088. else
  2089. gpg_import "${KEY_ID}" SEC
  2090. gpg_key_cache RESET "${KEY_ID}"
  2091. if gpg_sec_avail "${KEY_ID}"; then
  2092. GPG_KEY_SIGN="${KEY_ID}"
  2093. fi
  2094. fi
  2095.  
  2096. # interpret sign key setting
  2097. if var_isset 'GPG_KEY_SIGN'; then
  2098. echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
  2099. else
  2100. echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
  2101. GPG_KEY_SIGN='disabled'
  2102. fi
  2103. fi
  2104. else
  2105. KEY_ID="${GPG_KEY_SIGN}"
  2106. if ! gpg_sec_avail "${KEY_ID}"; then
  2107. inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
  2108. gpg_import "${KEY_ID}" SEC
  2109. gpg_key_cache RESET "${KEY_ID}"
  2110. gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
  2111. fi
  2112. fi
  2113.  
  2114. # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
  2115. # key enc can deal without, but might profit from gpg-agent
  2116. # if GPG_PW is not set alltogether
  2117. # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
  2118. if ! gpg_symmetric && \
  2119. ( ! var_isset GPG_PW || \
  2120. ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
  2121.  
  2122. if [ "$GPG_AGENT_ERR" -eq 1 ]; then
  2123. warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
  2124. elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
  2125. warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
  2126. elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
  2127. warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
  2128. else
  2129. echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
  2130. GPG_USEAGENT="--use-agent"
  2131. fi
  2132. fi
  2133.  
  2134. # end GPG config plausibility check2
  2135. fi
  2136.  
  2137. # config plausibility check - SPACE ###########################################
  2138.  
  2139. # is tmp is a folder and writable
  2140. CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
  2141. run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
  2142. if [ "$?" != "0" ]; then
  2143. error "Temporary file space '$TEMP_DIR' is not a directory or writable."
  2144. fi
  2145.  
  2146.  
  2147. # get volsize, default duplicity volume size is 25MB since v0.5.07
  2148. VOLSIZE=${VOLSIZE:-25}
  2149. # double if asynch is on
  2150. echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
  2151.  
  2152. # TODO: check for enough (async= upload space and WARN only
  2153. # use function tmp_space
  2154. #echo TODO: reimplent tmp space check
  2155.  
  2156.  
  2157. # test - GPG SANITY #####################################################################
  2158. # if encryption is disabled, skip this whole section
  2159. if gpg_disabled; then
  2160. echo -e "Test - En/Decryption skipped. (GPG disabled)"
  2161. elif [ "$GPG_TEST" = "disabled" ]; then
  2162. echo -e "Test - En/Decryption skipped. (Testing disabled)"
  2163. else
  2164.  
  2165. GPG_TEST="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
  2166. function cleanup_gpgtest {
  2167. echo -en "Cleanup - Delete '${GPG_TEST}_*'"
  2168. rm "${GPG_TEST}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
  2169. }
  2170.  
  2171. # signing enabled?
  2172. if gpg_signing; then
  2173. CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
  2174. CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
  2175. fi
  2176.  
  2177. # using keys
  2178. if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
  2179.  
  2180. for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
  2181. CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
  2182. done
  2183. # check encrypting
  2184. CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
  2185. run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST}_ENC") -e $(qw "$ME_LONG")
  2186. CMD_ERR=$?
  2187.  
  2188. if [ "$CMD_ERR" != "0" ]; then
  2189. KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
  2190. [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
  2191. 'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
  2192. can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
  2193. error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
  2194. fi
  2195.  
  2196. # check decrypting
  2197. CMD_MSG="Test - Decrypt"
  2198. gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
  2199. run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST}_ENC")
  2200. CMD_ERR=$?
  2201.  
  2202. if [ "$CMD_ERR" != "0" ]; then
  2203. error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
  2204. fi
  2205.  
  2206. # symmetric only
  2207. else
  2208. # check encrypting
  2209. CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
  2210. run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST}_ENC") --batch -c $(qw "$ME_LONG")
  2211. CMD_ERR=$?
  2212. if [ "$CMD_ERR" != "0" ]; then
  2213. error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
  2214. fi
  2215.  
  2216. # check decrypting
  2217. CMD_MSG="Test - Decryption with passphrase"
  2218. run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST}_DEC") --batch -d $(qw "${GPG_TEST}_ENC")
  2219. CMD_ERR=$?
  2220. if [ "$CMD_ERR" != "0" ]; then
  2221. error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
  2222. fi
  2223. fi
  2224.  
  2225. # compare original w/ decryptginal
  2226. CMD_MSG="Test - Compare"
  2227. [ -r "${GPG_TEST}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
  2228. run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST}_DEC')\""
  2229. CMD_ERR=$?
  2230. if [ "$CMD_ERR" = "0" ]; then
  2231. cleanup_gpgtest
  2232. else
  2233. error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
  2234. fi
  2235.  
  2236. fi # end disabled
  2237.  
  2238. ## an empty line
  2239. #echo
  2240.  
  2241. # Exclude file is needed, create it if necessary
  2242. [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
  2243.  
  2244. # export only used keys, if bkp not already exists ######################################
  2245. gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
  2246.  
  2247.  
  2248. # command execution #####################################################################
  2249.  
  2250. # urldecode url vars into plain text
  2251. var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
  2252. var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
  2253.  
  2254. # defined TARGET_USER&PASS vars replace their URL pendants
  2255. # (double defs already dealt with)
  2256. var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
  2257. var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
  2258.  
  2259. TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
  2260.  
  2261. # issue some warnings
  2262. case "$TARGET_URL_PROT_lowercase" in
  2263. 'cf+http')
  2264. # info on missing AUTH_URL
  2265. if ! var_isset 'CLOUDFILES_AUTHURL'; then
  2266. inform "No CLOUDFILES_AUTHURL exported (in conf).
  2267. Will use default which is probably rackspace."
  2268. fi
  2269.  ;;
  2270. 'swift')
  2271. # info on possibly missing AUTH_URL
  2272. var_isset 'SWIFT_AUTHURL' ||\
  2273. warning "\
  2274. Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
  2275.  ;;
  2276. 'rsync')
  2277. # everything in url (this backend does not support pass in env var)
  2278. # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
  2279. # print warning older version is detected
  2280. duplicity_version_lt 610 &&
  2281. warning "\
  2282. Duplicity version '$DUPL_VERSION' does not support providing the password as
  2283. env var for rsync backend. For security reasons you should consider to
  2284. update to a version greater than '0.6.10' of duplicity."
  2285.  ;;
  2286. esac
  2287.  
  2288.  
  2289. # for all protocols we put username in url and pass into env var
  2290. # for sec�rity reasons, we url_encode username to protect special chars
  2291. # first sortout backends with special ways to handle password
  2292. case "$TARGET_URL_PROT_lowercase" in
  2293. 'imap'|'imaps')
  2294. var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
  2295.  ;;
  2296. *)
  2297. # rest uses FTP_PASS var
  2298. var_isset 'TARGET_URL_PASS' && \
  2299. BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
  2300.  ;;
  2301. esac
  2302. # insert url encoded username into target url if needed
  2303. if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
  2304. BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
  2305. else
  2306. BACKEND_URL="$TARGET"
  2307. fi
  2308.  
  2309.  
  2310. # protect eval from special chars in url (e.g. open ')' in password,
  2311. # spaces in path, quotes) happens above in duplify() via quotewrap()
  2312. SOURCE="$SOURCE"
  2313. BACKEND_URL="$BACKEND_URL"
  2314. EXCLUDE="$EXCLUDE"
  2315. # since 0.7.03 --exclude-globbing-filelist is deprecated
  2316. EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist"
  2317.  
  2318. # translate backup to batch command
  2319. cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
  2320.  
  2321. # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
  2322. cmds=$(awk -v cmds="$cmds" "BEGIN{ \
  2323. gsub(/\+/,\"_and_\",cmds);\
  2324. gsub(/\-/,\"_or_\",cmds);\
  2325. gsub(/\[/,\"_groupIn_\",cmds);\
  2326. gsub(/\]/,\"_groupOut_\",cmds);\
  2327. print cmds}")
  2328. # convert cmds to array, lowercase for safety
  2329. declare -a CMDS
  2330. CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
  2331.  
  2332. unset FTPL_ERR
  2333.  
  2334. # run cmds
  2335. for cmd in ${CMDS[*]};
  2336. do
  2337.  
  2338. ## init
  2339. # raise index in cmd array for pre/post param
  2340. var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
  2341.  
  2342. unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
  2343.  
  2344. # get next cmd,cnd vars
  2345. nextno=$(( $CMD_NO ))
  2346. while ! var_isset 'CMD_NEXT'
  2347. do
  2348. nextno=$(($nextno+1))
  2349. if [ "$nextno" -lt "${#CMDS[@]}" ]; then
  2350. CMD_VALUE=${CMDS[$nextno]}
  2351. is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
  2352. is_groupMarker "$CMD_VALUE" && continue
  2353. CMD_NEXT="$CMD_VALUE"
  2354. else
  2355. CMD_NEXT='END'
  2356. fi
  2357. done
  2358.  
  2359. # get prev cnd, cnds are skipped pseudocmds
  2360. prevno=$(( $CMD_NO ));
  2361. while ! var_isset 'CND_PREV'
  2362. do
  2363. prevno=$(($prevno-1))
  2364. if [ "$prevno" -ge 0 ]; then
  2365. CMD_VALUE=${CMDS[$prevno]}
  2366. is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
  2367. is_command "$CMD_VALUE" && break
  2368. else
  2369. break
  2370. fi
  2371. done
  2372.  
  2373. # get prev cmd command minus skipped commands, only executed
  2374. prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
  2375. while ! var_isset 'CMD_PREV'
  2376. do
  2377. prevno=$(($prevno-1))
  2378. if [ "$prevno" -ge 0 ]; then
  2379. CMD_VALUE=${CMDS[$prevno]}
  2380. is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
  2381. is_groupMarker "$CMD_VALUE" && continue
  2382. CMD_PREV="$CMD_VALUE"
  2383. else
  2384. CMD_PREV='START'
  2385. fi
  2386. done
  2387.  
  2388. function get_cmd_skip_count {
  2389. # find closing bracket, get group skip count
  2390. local nextno=$CMD_NO
  2391. local GRP_OPEN=0
  2392. local GRP_SKIP=0
  2393. local CMD_VALUE
  2394. while [ "$nextno" -lt "${#CMDS[@]}" ]
  2395. do
  2396. nextno=$(($nextno+1))
  2397. CMD_VALUE=${CMDS[$nextno]}
  2398. GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
  2399. if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
  2400. break;
  2401. elif [ "$CMD_VALUE" == 'groupin' ]; then
  2402. GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
  2403. elif [ "$CMD_VALUE" == 'groupout' ]; then
  2404. GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
  2405. if [ "$GRP_OPEN" -lt 1 ]; then
  2406. break;
  2407. fi
  2408. fi
  2409. done
  2410.  
  2411. echo $GRP_SKIP;
  2412. }
  2413.  
  2414. # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
  2415. unset SKIP_NOW
  2416. if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
  2417. # skip cnd/grp cmds silently
  2418. is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
  2419. CMD_SKIP=$(($CMD_SKIP - 1))
  2420. SKIP_NOW="yes"
  2421. elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
  2422. CMD_SKIP=$(get_cmd_skip_count)
  2423. # incl. this "cmd"
  2424. CMD_SKIP=$(( $CMD_SKIP + 1 ))
  2425. unset CMD_SKIPPED
  2426. SKIP_NOW="yes"
  2427. elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
  2428. CMD_SKIP=$(get_cmd_skip_count)
  2429. # incl. this "cmd"
  2430. CMD_SKIP=$(( $CMD_SKIP + 1 ))
  2431. unset CMD_SKIPPED
  2432. SKIP_NOW="yes"
  2433. elif is_condition "$cmd" || is_groupMarker "$cmd"; then
  2434. unset 'CMD_SKIP';
  2435. SKIP_NOW="yes"
  2436. fi
  2437.  
  2438. # let's do the skip now
  2439. if [ -n "$SKIP_NOW" ]; then
  2440. # sum up how many commands we actually skipped for the prev var routines
  2441. CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
  2442. continue
  2443. fi
  2444.  
  2445. # save start time
  2446. RUN_START=$(nsecs)
  2447.  
  2448. # export some useful env vars for external scripts/programs to use
  2449. export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
  2450. TARGET_URL_USER TARGET_URL_PASS \
  2451. GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
  2452. GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
  2453. RUN_START
  2454.  
  2455. # user info
  2456. echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
  2457.  
  2458. case "$(tolower $cmd)" in
  2459. 'pre'|'post')
  2460. if [ "$cmd" == 'pre' ]; then
  2461. script=$PRE
  2462. else
  2463. script=$POST
  2464. fi
  2465. # script execution in a subshell, protect us from failures/var overwrites
  2466. ( run_script "$script" )
  2467. ;;
  2468. 'bkp')
  2469. duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
  2470. "$SOURCE" "$BACKEND_URL"
  2471. ;;
  2472. 'incr')
  2473. duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
  2474. "$SOURCE" "$BACKEND_URL"
  2475. ;;
  2476. 'full')
  2477. duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
  2478. "$SOURCE" "$BACKEND_URL"
  2479. ;;
  2480. 'verify')
  2481. TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
  2482. duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
  2483. "$BACKEND_URL" "$SOURCE"
  2484.  ;;
  2485. 'verifypath')
  2486. TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
  2487. IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
  2488. ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
  2489.  
  2490. Hint:
  2491. Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
  2492.  
  2493. duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
  2494. --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
  2495. ;;
  2496. 'list')
  2497. # time param exists since 0.5.10+
  2498. TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
  2499. duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
  2500.  ;;
  2501. 'cleanup')
  2502. duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
  2503.  ;;
  2504. 'purge')
  2505. MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
  2506. [ -z "$MAX_AGE" ] && error " Missing parameter <max_age>. Can be set in profile or as command line parameter."
  2507.  
  2508. duplify remove-older-than "${MAX_AGE}" \
  2509. -- "${dupl_opts[@]}" "$BACKEND_URL"
  2510.  ;;
  2511. 'purgefull')
  2512. MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
  2513. [ -z "$MAX_FULL_BACKUPS" ] && error " Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
  2514.  
  2515. duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
  2516. -- "${dupl_opts[@]}" "$BACKEND_URL"
  2517.  ;;
  2518. 'purgeincr')
  2519. MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
  2520. [ -z "$MAX_FULLS_WITH_INCRS" ] && error " Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
  2521.  
  2522. duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
  2523. -- "${dupl_opts[@]}" "$BACKEND_URL"
  2524.  ;;
  2525. 'restore')
  2526. OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
  2527. [ -z "$OUT_PATH" ] && error " Missing parameter target_path for restore.
  2528.  
  2529. Hint:
  2530. Syntax is -> $ME <profile> restore <target_path> [<age>]"
  2531.  
  2532. duplify -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
  2533.  ;;
  2534. 'fetch')
  2535. IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}";
  2536. TIME="${ftpl_pars[2]:-now}";
  2537. ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error " Missing parameter <src_path> or <target_path> for fetch.
  2538.  
  2539. Hint:
  2540. Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
  2541.  
  2542. # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
  2543. duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
  2544. --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
  2545.  ;;
  2546. 'status')
  2547. duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
  2548.  ;;
  2549. *)
  2550. error " Unknown command '$cmd'.
  2551.  
  2552. Hint:
  2553. Use '$ME usage' to get usage help."
  2554.  ;;
  2555. esac
  2556.  
  2557. CMD_ERR=$?
  2558. RUN_END=$(nsecs)
  2559. RUNTIME=$(( $RUN_END - $RUN_START ))
  2560.  
  2561. # print message on error; set error code
  2562. if [ "$CMD_ERR" -ne 0 ]; then
  2563. error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
  2564. FTPL_ERR=1
  2565. fi
  2566.  
  2567. separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
  2568. Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
  2569.  
  2570. done
  2571.  
  2572. exit ${FTPL_ERR}
  2573.