LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Inserting data into calendar apps automatically (sort of) (https://www.linuxquestions.org/questions/linux-software-2/inserting-data-into-calendar-apps-automatically-sort-of-926698/)

dedec0 01-31-2012 06:19 AM

Inserting data into calendar apps automatically (sort of)
 
Hello,

I am trying to import some dates into a calendar app (Evolution on Ubuntu, specifically; it should not matter) and I am having a few difficulties.

The dates of my interest are on a spreadsheet. This document has three columns: the date, a short and long description of each "event" (assume day events, like holydays). I can have this same data into other similar formats, like CSV (so it could be easiear to import it into other programs, apparently).

Until now, I couldn't find a way to import this data into Evolution. I can't find the exact format it expect a CSV to be (I have tried); nor I could find how to build a ICS file with the data, if not purely manual.

The main difficulty is that everything I have found (on web, etc.) assumes you are trying to import data that was exported from another calendar app (CSV, ICS, outlook format, etc.), wich is not the case.

So, what I am trying to find help to is: I have some data (CSV or spreadsheet, for simplification); this data is minimal, only contains what is of my interest (dates, assuming a whole day period); and I want to convert it to ICS format, so it can be accepted by Evolution and other calendar applications easily. And I want to do this conversion as automatic as possible.

catkin 01-31-2012 06:39 AM

OK -- what do you have in mind for a solution and what have you tried so far?

awk is a suitable language for the task. I have an awk script to parse a CSV file. It is complex because it accepts both commas and new lines in the data; maybe you don't need that ... ?

Once you have the data from the CSV, and know what each value is, writing an ICS is not complex unless you have repeating events.

dedec0 01-31-2012 06:58 AM

Well, catkin, I have tried:

- importing the CSV data (directly from the spreadsheet) into calendar/convertion apps. Sadly, they generate no events or null files, and give no error messages of any sort. :( I even tried different formats for the CSV file, for example, putting the date in the first column with different masks in the hopes it would be recognized as such.

- export some data from Evolution into a CSV file, and trying to see what it has. But it had too much "unusefull" data that made it hard for me to create a CSV from my data in a similar file.

- tried to find ready solutions (on web, programs, etc.) by searching and reading a little about ICS, iCalendar, related help files, etc.

In the end, I think that what I am trying to do is not really special. In my opinion, is sort of natural to have some data in a format similar to my spreadsheet. And then, there should be an easy way to have this data inserted in calendar apps (that not purely manual).

Your script maybe usefull. Can you attach it here?

catkin 01-31-2012 07:38 AM

The script that includes parsing a CSV is on pastebin.

What does your CSV look like?

dedec0 01-31-2012 07:52 AM

Like this example, separated with tabs and date format DD/MM/YYYY:

Code:

01/01/2012        "Day of bla 1"        "In the day of bla 1, bla bla bla bla bla bla..."
17/01/2012        "Day of bla 2"        "In the day of bla 2, bla bla bla bla bla bla..."
21/02/2012        "Day of bla 3"        "In the day of bla 3, bla bla bla bla bla bla..."
28/02/2012        "Day of bla 4"        "In the day of bla 4, bla bla bla bla bla bla..."
13/03/2012        "Day of bla 5"        "In the day of bla 5, bla bla bla bla bla bla..."
29/03/2012        "Day of bla 6"        "In the day of bla 6, bla bla bla bla bla bla..."
31/08/2012        "Day of bla 7"        "In the day of bla 7, bla bla bla bla bla bla..."


catkin 01-31-2012 10:51 PM

Quote:

Originally Posted by dedec0 (Post 4589323)
Like this example, separated with tabs and date format DD/MM/YYYY:

Thanks for that. Will there ever be any tabs or newlines in the blah parts of the data?

dedec0 02-01-2012 05:41 AM

Quote:

Originally Posted by catkin
Thanks for that. Will there ever be any tabs or newlines in the blah parts of the data?

No. No tabs or newlines is a restriction I can live with. :)

catkin 02-01-2012 06:23 AM

OK, then this awk script is good enough to parse the input:
Code:

#!/usr/bin/awk -f

BEGIN {
    FS = "\t"
}
{
    print "$1 is " $1
    print "$2 is " $2
    print "$3 is " $3
}

Make it executable (chmod 755 my_script.awk) and, when in its directory, call it by something like ./my_script.awk input.txt

If it works, the next step is to output to an ICS file. What is each field in ICS terminology?

dedec0 02-01-2012 07:35 AM

Really simple.

I ran your script and it works.

Quote:

Originally Posted by catkin
If it works, the next step is to output to an ICS file. What is each field in ICS terminology?

I don't know... :o

The first is a date (a whole day period). The second is a short description. The third is a long description.

What are the options?

catkin 02-01-2012 10:32 AM

RFC 5545 describes the ICS requirements. According to my reading of it, a minimal example of an ICS is something like this (you could try importing it into Evolution)
Code:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//decdec0//Experimental//EN
BEGIN:VEVENT
UID:20120201T203812Z-1000@decdec0.localdomain
CLASS:PUBLIC
DTSTAMP:20120201T203812Z
SUMMARY:Test event
DTSTART;20120222T140000
DURATION:PT30M
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR

Notes:
  1. Some lines may not be required. You could try omitting them and importing into Evolution. They are UID, CLASS and TRANSP (the last defaults to OPAQUE meaning "busy" - see below).
  2. UID must be unique. I designed it following RFC recommendations. It is <date>T<time>Z-<unique number>@decdec0.localdomain. <date> is YYYMMDD. <time> is HHMMSS. The script can increment <unique number> from 1000 upwards. decdec0.localdomain is intended to be unique to you.
  3. DTSTAMP is the time the calendar event was created as a UTC time. The script can use the same <date>T<time>Z for this and the UID.
  4. DTSTART is the start date in local time as <date>T<time>. The script can generate <date> from the first field of the CSV but you will need to add the time somewhere. Maybe no time implies just after midnight.
  5. DURATION is, er, the duration. The example is for 30 minutes. You will probably need either a DURATION or a DTEND (end date-time).
  6. TRANSP defines whether this is considered "busy" time (such as a meeting) or not (such as the Emperor of Mongolia's birthday unless you will attend the celebrations). If you don't want these events' periods to be considered "busy" the script will have to write TRANSP=TRANSPARENT.
It may be helpful to export an ICS from Evolution and compare it with the above example.

dedec0 02-01-2012 11:21 AM

Not to be lazy. But I did not want to go reading (or skimming) through the RFC. It seems that isn't going to be possible - no solution exists by now. I need less, and from a higher point of view.

What exactly should I expect Evolution to do with your example?

Yes, initially I tried to export from Evolution some ICS file. It was long and confuse to read.

Evolution initially seemed to like the data:
Quote:

Event
Classification Public
Resume Test event
Start 22-02-2012 14:00:00
Limit 22-02-2012 14:30:00
But I can't find it after importing! (maybe a bug in Evolution, maybe a bug in our data, maybe I am really stupid with Evolution...)

I am starting to give up with this idea. Simple spreadsheets seem better.

EDIT:

On a other tests, with trivial editions (dates and duration) on the ICS file, Evolution showed up with something. What a clumsy thing...

catkin 02-01-2012 11:37 AM

Understood -- the RFC is not light reading!

The sample ICS, when imported should create an event on 22 February 2012 at 2 PM with duration 30 minutes marked as "busy". I don't use Evolution but if you then create another event including the test event's time, say 22 February 2012 from 1 PM to 3 PM, it should warn (?) you of the event clash.

dedec0 02-01-2012 12:03 PM

Less is more
 
Code:

BEGIN:VCALENDAR
BEGIN:VEVENT
SUMMARY:Simpler test of all
DTSTART;20120228T000000
DURATION:PT24H
END:VEVENT
END:VCALENDAR

This worked (sometimes not exactly).

So, for many dates I would have many VEVENT's inside the VCALENDAR? Like, in xml'ish: "<cal><event></event></cal>" ?

And for the type of "event", are there simple options?

catkin 02-01-2012 12:12 PM

Good to know that something as simple as that worked. Now we are in a position to start developing the awk script to generate something like that from the CSV but it's late and I'm hungry and tired so no more from me today.

Meanwhile, could you try a DESCRIPTION as well as the SUMMARY? I'm thinking your third CSV field could go in it.

Will you be OK with all the DURATIONs being 24 hours?

catkin 02-01-2012 12:16 PM

Quote:

Originally Posted by dedec0 (Post 4590644)
This worked (sometimes not exactly).

So, for many dates I would have many VEVENT's inside the VCALENDAR? Like, in xml'ish: "<cal><event></event></cal>" ?

And for the type of "event", are there simple options?

What does "sometimes not exactly" mean? Does it matter to you?

Yes -- the BEGIN:VEVENT to END:VEVENT sequence can be repeated many times, as the Evolution ICS export should show.

What do you mean by "And for the type of "event", are there simple options?"? There are TODOs as well as VEVENTs and maybe more.


All times are GMT -5. The time now is 08:11 PM.