LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Blogs > jere21
User Name
Password

Notices


Rate this Entry

Sync calendar and contacts (ownCloud): General (Part 1)

Posted 07-14-2015 at 08:07 AM by jere21
Updated 07-14-2015 at 09:07 AM by jere21

This is the first of a series of blog posts with a complete guide to sync calendar and contacts between Debian Jessie laptop (as server and client) and Android (CyanogenMod) mobile:
  1. General
  2. Setup CalDAV/CardDAV server (on my laptop):
    ownCloud, Apache, self-signed certificate
  3. Setup calendar and contacts in Icedove (rebranded Thunderbird) and Iceowl (rebranded Lightning):
    export, transform and backup the data with a script (owncloud.export), SOGo connector, ThunderSync, print to paper
  4. Setup calendar and contacts on the Android mobile:
    F-Droid market, import certificate, DAVdroid
  5. Alternative Servers (NOT USED, NOT COMPLETE):
    Radicale, Calypso

  • tl;dr: Summary Outcome:

    • Calendar/Tasks: Read/Write access in all clients (ownCloud webinterface, Iceowl, Android aCalendar/Tasks).
    • Contacts:
      • Read access in all clients, but Icedove doesn't display all information.
      • Write access in ownCloud webinterface.
      • No write access in Icedove (doesn't matter too much since both Icedove and ownCloud server are on localhost and thus always work together, even if there is no net at all).
      • Write access in Android Contacts works, but it changes some field names (in most cases still understood and displayed correctly by ownCloud, just in a few "exotic" cases the field sub-type changes). AFAICS this doesn't loose you any data. So I use it, but avoid "exotic" field types in all clients.
    • Automatic sync:
      • ownCloud webinterface: instantly, but a bit slow (you'll notice this when you do several changes and a field disappears or shows its old value. No big issue for practical usage).
      • Automatic sync after change in client, or automatic every 5-60 minutes.
      • Icedove addressbook (read-only): on Icedove start automatic import of automatically exported data.
    • Both mobile and laptop have to be online in my (or an identically named) home network to sync.
    • Backup:
      • Exports are stored regularly and automatically in git.
      • The contacts can be easily printed on paper.

  • Goals:

    • Sync calendar, tasks and addressbook between the clients on the laptop (Debian Jessie with Icedove 31 + Iceowl extension as clients) and Android mobile (Android 4.3.1/CyanogenMod 10.2.1-galaxysmtd with aCalendar, Tasks, and Contacts).
    • Sync without giving the data to third partys (Google), so I also need a server (running on my laptop).
    • Store the database or an export of it in version control (locally). This helps to diagnose if and where data gets lost (be prepared, some client definitely will do it) and to restore the data, even if you don't realize immediately that some data was lost.
    • Do this only in the local wireless home network. This is more secure, but at the same time not that convenient.

  • Solution:

    This means CalDAV (IETF standard since 2007) for the calendar and tasks, and CardDAV (IETF standard since 2011) for the contacts/addressbook.

    I started this adventure years ago with funambol. It's not packaged, and uses its own Java and Tomcat, so I never felt comfortable to rely on it and didn't set up a real working solution.

    Later I tried SOGo. But I stopped when I wanted to setup my Android phone and saw that the recommended additions CalDAV-Sync and CardDAV-Sync unfortunately are (were?) not Open Source (FLOSS).

    This time I first tried radicale as server, which worked just out of the box. It's a lightweigt, designated solution for a CalDAV- and CardDAV server. I especially liked that it allows to handle its database with git. I gave up on it when I encountered problems syncing my Icedove addressbook. You'll see my inital notes about this and its fork calypso in post 4.

    CalDAV/CardDAV server (running on my laptop):

    So I moved on to ownCloud, where I encountered the same Icedove sync problems, but solved them. While this solution would also work for radicale, I'm very happy with this decision, because ownCloud is general purpose and allows to move further services off the cloud. Besides it has a much bigger popcon statistic in Debian, which will make live probably much easier in the future.

    Android (CyanogenMod) Phone:

    The real big change to my last try is that there's finally a real Open Source app: DAVdroid, a CalDAV/CardDAV synchronisation adapter which makes the contacts and calendars (events) apps work with my own server instead of relying on Google).

    Icedove/Iceowl calendar:

    I use the packaged SOGo connector to sync my new cloud calendar(s) in Iceowl both ways. This works fine, except that the contact's birthdays aren't shown in Iceowl.

    Icedove contacts:

    Because there were more critical issues with contacts syncing using the SOGo connector, I came up with exporting the addressbook from ownCloud and preparing it for Icedove (both automatically with my own script owncloud.export), and then importing it automatically in Icedove with the manually installed addon ThunderSync.
    ThunderSync was written for exporting the Thunderbird Contacts. But it does a great (yet not perfect job) for importing vCard files, too. You can assign it to any Icedove addressbook.
    I use it to automatically import ownCloud's exported contacts on Icedove's start. Thus I have a Icedove addressbook that is synced read-only from ownCloud. There's absolutely no risk for the ownCloud data because this happens in several separate one-way steps. If Icedove messes up the addressbook it doesn't matter, because the next Icedove start will give back the ownCloud addressbook.
    To add an address in Icedove (only) I might use the separate "Personal Addressbook" (permanent local store, but no sync back to ownCloud). So I just add/change everything directly in ownCloud's webinterface.
    ThunderSync only supports vCard 2.1, while ownCloud exports vCard 3.0. owncloud.export takes care of some cases where Icedove has a field defined, but the contact data isn't imported.

    Data backup:

    owncloud.export exports my calendar and contacts and stores them in git. It also prepares a csv file of the contacts which can be printed directly. (Printing on paper is the real backup, so now I'm also ready to drop my old, handwritten paper addressbooks in favor of a digital one.)

  • Issues/Lessons learned:

    • Always import old data directly in ownCloud (using the web interface). Not all clients support all fields (even if they didn't have bugs). Check your imported data if it is complete.
    • Sometimes performance is an issue on the web interface, especially during mass editing of contacts. Sometimes a page reload helps, but e.g. the "preferred" tick sometimes is forgotten or the type is changed to preferred and the original type (home, work, ...) is (temporarily?) forgotten. This might be related to my choice of SQLite. Check owncloud.export's commits with e.g. gitk to see what's really stored in ownCloud's database.
    • If possible always tick one entry as preferred. owncloud.export and some clients use it (while others just show/emphasize the first or last line if there are several entries).
    • The ownCloud webinterface and most other clients always show the first entry, if there are several candidates for the same field, ignoring the preferred flag. (Icedove displays the last entry and ignores the preferred flag). You can manually sort the exported Contacts vCard, and reimport it as a new addressbook.
    • The calendar exchange between Icedove and Android aCalendar (for events created on each of them with or withhout reminder) seems to work flawlessly.
    • Editing an existing contact in Android Contacts and syncing back to ownCloud:
      • Android Contact's ignores ownCloud's FN (single field entry for the displayed name), and uses N (multiple fields for surname, first name, ...) instead to construct a new FN (first name and surname) which gets synced back then. Normally the result is the same again.
      • The main category fields are sorted together, just as they are presented on your mobile (BEGIN, VERSION, X-MOZILLA-HTML, FN, N, TEL, EMAIL, IMPP, ADR, LABEL (is added to every ADR and thus duplicates the data internally), CATEGORIES, BDAY, PRODID, REV, END). (Sub-)types or the data itself stay in the order they were previously. Type values are changed from upper- to lower-case. This mostly changes the data just internally, without changing it's display, so no big issue.
      • The ownCloud's Instant Messaging subtypes "Jabber", "Google Talk", "XMPP" and "Facebook" are all displayed as subtype "XMPP" in ownCloud after the roundtrip.
      • "Yahoo", "QQ", "GaduGadu" and "ownCloud" are displayed without a subtype in the "Instant messaging" section after the roundtrip.
    • Issues why I don't sync contacts in Icedove with the SOGo connector addon:
    • Hints for debugging failed login
      /var/log/owncloud.log:
      Code:
      {"reqId":"558236a087820","app":"core","message":"Login failed: 'jens' (Remote IP: '192.168.178.35', X-Forwarded-For: '')","level":2,"time":"2015-06-18T03:10:24+00:00","method":"PROPFIND","url":"\/owncloud\/"}
    • [I'm not sure if I correctly understood the to-be-used well-known URL, but:] The well-known URL https://hope.fritz.box/owncloud doesn't work, although it seems to be implemented in /etc/owncloud/htaccess (linked to by /usr/share/owncloud/.htaccess):
      Code:
      cat /etc/owncloud/htaccess
      ###[...]
      ###<IfModule mod_rewrite.c>
      ###RewriteEngine on
      ###RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
      ###RewriteRule ^/owncloud/\.well-known/host-meta /owncloud/public.php?service=host-meta [QSA,L]
      ###RewriteRule ^/owncloud/\.well-known/host-meta\.json /owncloud/public.php?service=host-meta-json [QSA,L]
      ###RewriteRule ^/owncloud/\.well-known/carddav /owncloud/remote.php/carddav/ [R]
      ###RewriteRule ^/owncloud/\.well-known/caldav /owncloud/remote.php/caldav/ [R]
      ###RewriteRule ^/owncloud/apps/calendar/caldav\.php /owncloud/remote.php/caldav/ [QSA,L]
      ###RewriteRule ^/owncloud/apps/contacts/carddav\.php /owncloud/remote.php/carddav/ [QSA,L]
      ###RewriteRule ^/owncloud/remote/(.*) /owncloud/remote.php [QSA,L]
      ###</IfModule>
      ###[...]
      cat /etc/apache2/mods-enabled/rewrite.load 
      ###LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so
      Using the well-known URL https://hope.fritz.box/owncloud in DAVdroid doesn't work. I get in /var/log/apache2/access.log:
      Code:
      192.168.178.35 - - [18/Jun/2015:17:01:36 +0200] "PROPFIND /.well-known/carddav HTTP/1.1" 405 2222 "-" "DAVdroid/0.8.0"
      192.168.178.35 - - [18/Jun/2015:17:01:37 +0200] "PROPFIND /owncloud HTTP/1.1" 301 859 "-" "DAVdroid/0.8.0"
      192.168.178.35 - jens [18/Jun/2015:17:01:37 +0200] "PROPFIND /owncloud/ HTTP/1.1" 405 933 "-" "DAVdroid/0.8.0"
      192.168.178.35 - jens [18/Jun/2015:17:01:37 +0200] "PROPFIND /owncloud/ HTTP/1.1" 405 1094 "-" "DAVdroid/0.8.0"
      192.168.178.35 - - [18/Jun/2015:17:01:37 +0200] "PROPFIND /.well-known/caldav HTTP/1.1" 405 843 "-" "DAVdroid/0.8.0"
      192.168.178.35 - - [18/Jun/2015:17:01:37 +0200] "PROPFIND /owncloud HTTP/1.1" 301 859 "-" "DAVdroid/0.8.0"
      192.168.178.35 - jens [18/Jun/2015:17:01:37 +0200] "PROPFIND /owncloud/ HTTP/1.1" 405 933 "-" "DAVdroid/0.8.0"
      192.168.178.35 - jens [18/Jun/2015:17:01:38 +0200] "PROPFIND /owncloud/ HTTP/1.1" 405 1094 "-" "DAVdroid/0.8.0"
      And when using the "well-known" URL in Iceowl this:
      Code:
      127.0.0.1 - - [18/Jun/2015:17:10:13 +0200] "PROPFIND /owncloud/ HTTP/1.1" 405 2539 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.7.0 Lightning/3.3.7"
    • There is also a /etc/owncloud/.htaccess file (note the dot), unused!?
Posted in Uncategorized
Views 1565 Comments 0
« Prev     Main     Next »
Total Comments 0

Comments

 

  



All times are GMT -5. The time now is 08:58 AM.

Main Menu
Advertisement
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration