LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Blogs > jere21
User Name
Password

Notices


Rate this Entry

Sync calendar and contacts (ownCloud): Clients on laptop (Part 3)

Posted 07-14-2015 at 09:12 AM by jere21
Updated 10-28-2015 at 12:34 PM by jere21

This is the third of a series of blog posts with a complete guide to sync calendar and contacts between Debian Jessie laptop (as server and client) and Android (CyanogenMod) mobile:
  1. General
  2. Setup CalDAV/CardDAV server (on my laptop):
    ownCloud, Apache, self-signed certificate
  3. Setup calendar and contacts in Icedove (rebranded Thunderbird) and Iceowl (rebranded Lightning):
    export, transform and backup the data with a script (owncloud.export), SOGo connector, ThunderSync, print to paper
  4. Setup calendar and contacts on the Android mobile:
    F-Droid market, import certificate, DAVdroid
  5. Alternative Servers (NOT USED, NOT COMPLETE):
    Radicale, Calypso


This is for the clients on the same machine as the ownCloud server.
  1. Icedove, Iceowl extension and Sogo connector

    Code:
    ii  icedove                 31.7.0-1  mail/news client with RSS and integrated spam filter support
    ii  iceowl-extension        31.7.0-1  Calendar Extension for Thunderbird/Icedove
    ii  xul-ext-sogo-connector  31.0.1-1  transforms Icedove into a full DAV client for groupware servers
    Done: Bug#686206: [icedove] Please clarify the relation between iceowl-extension, lightning and sunbird.

    See https://wiki.debian.org/SOGoConnector.

    • Create a new calendar:

      Icedove - Events&Tasks - Calendar
      Right-click in the left column in free space
      New calendar
      On the Network
      Format: CalDav
      Location: https://localhost/owncloud/remote.ph.../jens/personal (ownCloud's CalDAV link)
      Name: "ownCloud jens" (choose freely)
      Refresh calendar every 5 minutes.
      Don't tick offline support, since Calendar is on localhost (Not sure what this exactly does).
    • Add further calendars the same way. It works here with 2 calendars. But a common URL like "https://hope.fritz.box/owncloud/remote.php/caldav/" does not work (see DAVdroid on Android where this works to add all calendars).
    • Enable "Calendar Name" column to be shown.
      Icedove - Events and Tasks - Tasks
      Click in the right end of the Task overview menu and tick "Calendar Name"

  2. owncloud.export - handle exports from ownCloud

    owncloud.export does the following tasks:
    • Export calendars and contacts from ownCloud.
    • Store the exports in git.
    • Prepare a csv for printing the contacts on paper.
    • Prepare the contacts for ThunderSync import.

    Setup:

    1. Copy the following text, adjust the variables (loginnames and paths in lines 17-30) and change the "for" line (line 39). Then save it as /usr/local/bin/owncloud.export:
      Code:
      #!/bin/bash
      # owncloud.export - handle exports from ownCloud
      # - Export calendars and contacts from ownCloud.
      # - Store the exports in git.
      # - Prepare a csv for printing the contacts on paper.
      # - Prepare the contacts for ThunderSync import.
      
      
      # (C) 2015 by jere21@linuxquestions.org
      # Do whatever you want with this.
      
      set -e
      
      #
      # Edit the following variables
      #
      USER_OWNCLOUD="jens"
      USER_LOCAL="jens"
      PASSWORD="XXXXXXXX" # Saving plaintext passwords is awesome!
      # The URL is shown in ownCloud's Contacts/Calendar page if you hover over
      # the "Export" icon.
      # Fill in the URIs here and adjust the "for" line below.
      # Calendar variable naming: name_[URL|LOCAL] (choose name freely)
      # Contacts variable naming: CONTACTS_[URL|LOCAL] (do not change naming!)
      CALENDAR1_URL="https://localhost/owncloud/index.php/apps/calendar/export.php?calid=2"
      CALENDAR1_LOCAL="/home/jens/Documents/Privat/Calendar.git/Personal.ics"
      CALENDAR2_URL="https://localhost/owncloud/index.php/apps/calendar/export.php?calid=5"
      CALENDAR2_LOCAL="/home/jens/Documents/Privat/Calendar.git/Birthdays.ics"
      CONTACTS_URL="https://localhost/owncloud/index.php/apps/contacts/addressbook/local/5/export"
      CONTACTS_LOCAL="/home/jens/Documents/Privat/Contacts.git/Contacts.vcf"
      
      
      # For at least a littlebit security for the plaintext password do a
      #  chmod 700 owncloud.export
      #  chown ${USER_LOCAL}.${USER_LOCAL} owncloud.export.
      
      
      # If you have several calendars add them here:
      for i in CALENDAR1 CALENDAR2 CONTACTS ; do
      
        # Set LOCAL and URL for the above given url/path
        eval LOCAL="\$${i}_LOCAL"
        eval URL="\$${i}_URL"
      
        if [ "$1" = init ] ; then
          mkdir -p "$(dirname $LOCAL)"
          cd "$(dirname $LOCAL)"
          git init
          if [ "$i" = CONTACTS ] ; then
            cat > .gitignore << EOF
      *~
      *.swp
      \#*# 
      .~lock*                                                                                                                                                         
      $(basename $LOCAL .vcf).csv 
      $(basename $LOCAL .vcf).ods
      $(basename $LOCAL .vcf).txt 
      $(basename $LOCAL .vcf).aux 
      $(basename $LOCAL .vcf).log 
      $(basename $LOCAL .vcf).pdf
      $(basename $LOCAL .vcf)-book.pdf          
      EOF
            git add .gitignore
            git commit -m "Added .gitignore" || true
          fi
          cd
        fi
      
        # Export from ownCloud:
        # TODO: although we use localhost here, it would be nice to check the
        # certificate. But this seems not to be trivial for a self-signed
        # certificate.
        wget \
          --auth-no-challenge \
          --no-check-certificate \
          --user="$USER_OWNCLOUD" \
          --password="$PASSWORD" \
          -O "$LOCAL" \
          "$URL"
      
        cd "$(dirname $LOCAL)"
      
        # Add to git before changing anything:
        git add "$(basename $LOCAL)"
        git commit -m "ownCloud export $(date -Iminutes -u)" || true
      
        if [ "$i" = CONTACTS ] ; then
      
          # Variant 1:
          # Prepare csv for printing on paper:
          # Ignore unnecessary lines
          # Replace csv deliminiter ";" and "\n" with blanks
          # Merge duplicate spaces
          # Replace the leading blank (may happen when a value is very long and therefore wrapped) and
          # the first occurrence of ":" (vCard delimiter) with ";"
          # In the end there should be a csv with 2 columns.
          grep -Ev "^BEGIN|^VERSION|^UID|^REV|^PRODID|^END" "$(basename $LOCAL)" | \
            sed \
            -e "s|;| |g" \
            -e "s|\\\n| |g" \
            -e "s|  *| |g" \
            -e "s|^ |;|" \
            -e "s|:|;|" \
            > "$(basename $LOCAL .vcf).csv"
      
          # Variant 2:
          # Prepare txt for pdf for printing on paper:
          # Ignore unnecessary lines
          # Replace csv deliminiter ";" and "\n" with blanks
          # Merge duplicate spaces
          # Replace the leading blank (may happen when a value is very long and therefore wrapped) with a ":" and
          # Remove fieldnames (everything up to the first ":")
          grep -Ev "^BEGIN|^VERSION|^UID|^REV|^PRODID|^END" "$(basename $LOCAL)" | \
            sed \
            -e "s|;| |g" \
            -e "s|\\\n| |g" \
            -e "s|  *| |g" \
            -e "s|^ |:|" \
            -e "s|[^:]*:||" \
            > "$(basename $LOCAL .vcf).txt"
          # Create pdf
          xelatex Contacts.tex
          # Create pdf book (2 pages on front and 2 on back of DIN A4 paper)
          pdfbook Contacts.pdf
      
          # Prepare for ThunderSync import:
          # Adjust some field names so that ThunderSync can interpret them (hacky):
          # - Use sed to declare all email-addresses as SECONDARY and change back
          #   the PREFerred ones as PRIMARY.
          #   The only drawback is that if you have only one address, this will be
          #   imported as additional e-mail.
          #   But you'll still have the full functionality afaics.
          # - Adjust some Instant messenger field names.
          # - Take care of ORGanizations, where no department was specified.
          #   (Works only with bash, not dash)
          # - TODO: copy group to organization because Thunderbird does not use ORG(?)
          #
          # Hint for adding further additions:
          # - Check uncommitted changes in gitk after adding the missing value in the
          #   Thunderbird/Icedove ownCloud addressbook and ThunderSync exporting it.
          # - All current Owncloud names for the given value EMAIL:
          #     grep EMAIL Contacts.vcf |sed "s|:.*|:|g"|sort -u
          sed -i \
            -e "s|\(EMAIL;.*\):|\1;SECONDARY:|;" \
            -e "s|\(EMAIL;.*PREF.*\);SECONDARY:|\1;PRIMARY:|;" \
            -e "s|\(EMAIL;.*pref.*\);SECONDARY:|\1;PRIMARY:|;" \
            -e "s|IMPP;X-SERVICE-TYPE=googletalk:xmpp:|X-GOOGLE-TALK:|;" \
            -e "s|IMPP:xmpp:|X-JABBER:|;" \
            -e "s|IMPP;X-SERVICE-TYPE=xmpp:xmpp:|X-JABBER:|;" \
            -e "s|\(ORG:[[:alnum:]]*\)|\1;|g" \
            "$(basename $LOCAL)"
        fi
      
        chown -R ${USER_LOCAL}.${USER_LOCAL} "$(dirname $LOCAL)"
        cd
      done
    2. Restrict access to it because it contains your plaintext owncloud password - awesome :-/
      Code:
      chmod 700 owncloud.export
      chown jens.jens owncloud.export
    3. Copy the following text and save it as ~/Documents/Privat/Contacts.git/Contacts.tex:
      Code:
      \documentclass[a5paper]{article}
      \usepackage{fontspec}
      \usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm,foot=0.2cm]{geometry}
      \usepackage{listings}
      
      \lstset{ %
      basicstyle=\ttfamily\scriptsize,
      frame=none,
      keepspaces=true,
      }
      
      \begin{document}
      \lstinputlisting{Contacts.txt}
      \end{document}
    4. Install some TeX packages:
      Code:
      ii  texlive-extra-utils                2015.20151016-1
      ii  texlive-xetex                      2015.20151016-1
    5. Do the first run with
      Code:
      owncloud.export init
      . You can always repeat this step with no harm. This creates the configured folders for storing the exports, creates the git repository and adds a .gitignore file.

    6. Let owncloud.export run every hour with a systemd timer:

      See the man pages systemd.[timer|service|unit], and e.g. these two posts for documented examples.

      1. Prepare system:

        Code:
        mkdir -p $HOME/.config/systemd/user/
      2. $HOME/.config/systemd/user/owncloud.export.service:
        Code:
        [Unit]
        Description=owncloud.export - handle exports from ownCloud
        Requisite=apache2
        
        [Service]
        Type=simple
        ExecStart=/usr/local/bin/owncloud.export
        
        [Install]
        WantedBy=basic.target
        Note Type=simple vs. oneshot:
        Note that it is important to set the Type variable to be “simple”, not “oneshot”. Using “oneshot” makes it so that the script will be run the first time, and then systemd thinks that you don’t want to run it again, and will turn off the timer we make next.

      3. $HOME/.config/systemd/user/owncloud.export.timer:
        Code:
        [Unit]
        Description=Run owncloud.export(.service) regularly
        
        [Timer]
        # Time to wait after booting before we run first time
        OnBootSec=1min
        # Time between running each consecutive time
        OnUnitActiveSec=30min
        Unit=owncloud.export.service
        
        [Install]
        WantedBy=owncloud.export.service
      4. Enable them:

        Configure the timer to be automatically started at boot time:
        Code:
        systemctl --user enable owncloud.export.service 
        ###Created symlink from /home/jens/.config/systemd/user/basic.target.wants/owncloud.export.service to /home/jens/.config/systemd/user/owncloud.export.service.
        systemctl --user enable owncloud.export.timer
        ###Created symlink from /home/jens/.config/systemd/user/owncloud.export.service.wants/owncloud.export.timer to /home/jens/.config/systemd/user/owncloud.export.timer.
        Start the timer immediately:
        Code:
        systemctl --user start owncloud.export.timer
      5. Check if it is working:

        Code:
        systemctl --user list-timers
        ###NEXT                          LEFT       LAST                          PASSED  UNIT                  ACTIVATES
        ###Fri 2015-07-10 12:43:41 CEST  29min left Fri 2015-07-10 12:13:41 CEST  12s ago owncloud.export.timer owncloud.export.service
        ###
        ###1 timers listed.
        ###Pass --all to see loaded but inactive timers, too.

  3. Print your contacts as a real backup on paper:
    1. Variant 1:
      1. Execute owncloud.export
      2. LibreOffice Calc - New Spreadsheet
      3. Insert - Sheet From File - Contacts.csv
        • Character Set: UTF-8
        • Text-Delimiters: ;
      4. Format it:
        • Font: Liberation Sans
        • Size: 10
        • Page format:
          • DIN A4
          • Portrait
          • 0,8 cm border (left and right)
      5. After changing stuff apply "Optimal column width" to all columns.
      6. There should be 2 columns: 6,49 cm and 12,7 cm here. The total width of 19,2 cm fits in a DIN A4 (210 × 297 mm) page with 0,8 cm borders around.
      7. I print 4 pages per sheet with "Draw a border around each box".
    2. Variant 2:
      Simply print Contacts-book.pdf. Take care to insert papers correctly to your printer.
  4. ThunderSync

    I installed ThunderSync 2.3 from https://addons.mozilla.org/en-US/thu...n/thundersync/.

    Setup ThunderSync:

    1. Icedove - Tools - Addressbook
    2. File - New - Address Book...
      "ownCloud (auto-imported)"
    3. Right click on the new addressbook "ownCloud (auto-imported)": Thundersync - Configure...
      Address Book: "ownCloud (auto-imported)"
      General:
      Export format: vCard 2.1 (*.vcf, single file) ACCEPT WARNING
      file:///home/jens/Documents/Privat/Contacts.git/Contacts.vcf
      Start-up action: Import in Thunderbird
      Standard action: Import in Thunderbird
      Export Format
      File charset (Import): Unicode (UTF-8)
      Filter
      popularity index: ignore
    4. Either restart Icedove or get the contacts manually: Right click again on the addressbook "ownCloud (auto-imported)": Thundersync - Import this addressbook... - Confirm with "Synchronise"

    Note: The Contacts.vcf is also updated on "Import this addressbook..." (and "<<" is selected for a subvalue). Everything not imported just gets removed from it. See the uncommitted changes in Contacts.git after importing the Contacts.
    This doesn't matter here because my setup is working one-way and overwrites previous changes. The git commit happens after a fresh vcf arrives at this position and before the file gets crippled.


Edit 2015-07-15: Need to also enable the service (not just the timer), for the timer to start automatically at boot.
Edit 2015-10-28: let owncloud.export create a Contacts-book.pdf for efficient printing on paper.
Posted in Uncategorized
Views 1721 Comments 0
« Prev     Main     Next »
Total Comments 0

Comments

 

  



All times are GMT -5. The time now is 02:17 PM.

Main Menu
Advertisement
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration