LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   Evolution lost folders/emails after upgrading from FC14+GNOME to FC17+KDE (http://www.linuxquestions.org/questions/linux-software-2/evolution-lost-folders-emails-after-upgrading-from-fc14-gnome-to-fc17-kde-949338/)

kaza 06-09-2012 04:46 AM

Evolution lost folders/emails after upgrading from FC14+GNOME to FC17+KDE
 
Hello,

I upgraded my FC14+GNOME system to FC17+KDE.
Some things are not yet behaving as before and the most
annoying of them is what happened to Evolution:
When I open it on the new system, instead of showing MANY e-mail
folders with MANY e-mails in them only a few folders are whown
on the left panel (with "_sbd" appended to their name), most
of the folders I knew existed are not shown at all and those
that are shown - emty. Before the upgrade I did few backups
to tape, one of the entire system (from "/" and down), another
from "/home/user" and down and another of "/home/user/.*"
files and directories. It didn't occured to me to use the
evolutions own backup and restore as I didn't expect
this particular app to be more problematic than any other app.
But now, after the upgrade and encountering the problems
with evolution I first attempted to restore the

~/.local/share/evolution

directory. It didn't work. So I searched the web, found
(at http://library.gnome.org/users/evolution/3.2/data-storage.html.en)
the list of directories where evolution stores its data and attempted
to restore them:

Code:

<.../user>/bin/rm -fr .local/share/evolution
<.../user>/bin/rm -fr .config/evolution
<.../user>/bin/rm -fr .cache/evolution
<.../user>/bin/rm -fr .gconf/apps/evolution
<.../user>cp -r /home/user/temp/home_restore_from_backup_2012_06_02/.local/share/evolution .local/share
<.../user>cp -r /home/user/temp/home_restore_from_backup_2012_06_02/.config/evolution .config
<.../user>cp -r /home/user/temp/home_restore_from_backup_2012_06_02/.cache/evolution .cache
<.../user>cp -r /home/user/temp/home_restore_from_backup_2012_06_02/.gconf/apps/evolution .gconf/apps

After runninh the above, the only thing that got slightly better was
that the listing of the "Inbox" folder showed the many e-mails
that were there before the upgrade. But clicking on any one of them
resulted in an error message like:

Cannot get message 6413 from folder /home/user/.local/share/evolution/mail/local/.: No such file or directory

It seems to me that the indexing of messages by some numbers
is still wrong. And, after closing evolution and re-opening it
even the list of e-mail from the "Inbox" folder disappeared.

Also, when opening the evolution for the first time after the
upgrade (or after restoring the directories as above) I had
to enter my user password to access the keyring.

When I look at the contents of the

~/.local/share/evolution/mail/local

directory I see quartuplets of files named according to the
mail folders I used to have and the size of them shows they're
not empty:

Code:

total 1045756
-rw-r--r--. 1 user user  23895164 Jul 20 2010 21:13:30 buff
-rw-r--r--. 1 user user        53 Jul 20 2010 22:05:35 buff.cmeta
-rw-------. 1 user user    93184 Jul 18 2010 22:09:32 buff.ibex.index
-rw-------. 1 user user    48632 Jul 17 2010 20:38:28 buff.ibex.index.data
-rw-r--r--. 1 user user  2544761 Oct 13 2010 21:57:33 computer
-rw-r--r--. 1 user user        13 May 19 2012 20:49:20 computer.cmeta
-rw-------. 1 user user    60416 Oct 13 2010 22:00:07 computer.ibex.index
-rw-------. 1 user user    53200 Oct 13 2010 22:00:07 computer.ibex.index.data
drwx------. 2 user user      4096 Jun 09 2012 10:47:46 cur
-rw-------. 1 user user    10197 May 31 2012 07:23:02 Drafts
-rw-r--r--. 1 user user        13 Jun 05 2012 14:21:22 Drafts.cmeta
-rw-------. 1 user user    10240 May 31 2012 07:23:57 Drafts.ibex.index
-rw-------. 1 user user      1436 May 31 2012 07:23:57 Drafts.ibex.index.data
-rw-r--r--. 1 user user    371382 Nov 11 2011 21:47:53 EE
-rw-r--r--. 1 user user        13 Feb 06 2012 18:33:16 EE.cmeta
-rw-------. 1 user user    38912 Feb 05 2012 22:11:29 EE.ibex.index
-rw-------. 1 user user    29896 Feb 05 2012 22:11:29 EE.ibex.index.data
-rw-r--r--. 1 user user    79296 Mar 03 2011 19:45:49 ENERGY
-rw-r--r--. 1 user user        13 Mar 03 2011 19:46:01 ENERGY.cmeta
-rw-------. 1 user user    10240 Mar 03 2011 19:46:01 ENERGY.ibex.index
-rw-------. 1 user user      1316 Mar 03 2011 19:46:01 ENERGY.ibex.index.data
-rw-r--r--. 1 user user    45785 Mar 03 2011 19:45:56 engineering
-rw-r--r--. 1 user user        13 Mar 03 2011 19:46:01 engineering.cmeta
-rw-------. 1 user user    12288 Mar 02 2011 20:42:49 engineering.ibex.index
-rw-------. 1 user user      2552 Mar 02 2011 20:42:49 engineering.ibex.index.data
-rw-r--r--. 1 user user    31599 Jun 13 2010 22:16:59 fb
-rw-r--r--. 1 user user        13 Oct 23 2011 22:46:33 fb.cmeta
-rw-------. 1 user user    13312 Jun 13 2010 23:19:46 fb.ibex.index
-rw-------. 1 user user      3304 Jun 13 2010 23:19:46 fb.ibex.index.data
-rw-r--r--. 1 user user    622360 Sep 16 2011 16:13:53 finance
-rw-r--r--. 1 user user        13 Jan 09 2012 12:52:18 finance.cmeta
-rw-------. 1 user user    26624 Jan 09 2012 12:51:00 finance.ibex.index
-rw-------. 1 user user    12716 Jan 09 2012 12:51:00 finance.ibex.index.data
-rw-r--r--. 1 user user  6144000 Jun 09 2012 11:03:11 folders.db
-rw-r--r--. 1 user user  4041530 May 31 2012 06:53:43 GADGETS
-rw-r--r--. 1 user user        13 May 31 2012 07:23:57 GADGETS.cmeta
-rw-------. 1 user user    97280 May 31 2012 07:23:57 GADGETS.ibex.index
-rw-------. 1 user user    136896 May 31 2012 07:23:57 GADGETS.ibex.index.data
-rw-r--r--. 1 user user      6460 Apr 21 2012 16:30:42 going_out
-rw-r--r--. 1 user user        13 Apr 21 2012 16:32:17 going_out.cmeta
-rw-------. 1 user user    12288 Apr 21 2012 16:32:17 going_out.ibex.index
-rw-------. 1 user user      2016 Apr 21 2012 16:32:17 going_out.ibex.index.data
-rw-r--r--. 1 user user  2083595 May 23 2012 20:42:30 health
-rw-r--r--. 1 user user        13 May 23 2012 20:48:17 health.cmeta
-rw-------. 1 user user    43008 May 23 2012 20:48:17 health.ibex.index
-rw-------. 1 user user    25832 May 23 2012 20:48:17 health.ibex.index.data
-rw-r--r--. 1 user user  1295534 Nov 29 2011 19:42:40 House
-rw-r--r--. 1 user user        13 Apr 24 2012 00:47:57 House.cmeta
-rw-------. 1 user user    70656 Nov 29 2011 19:47:52 House.ibex.index
-rw-------. 1 user user    79104 Nov 29 2011 19:47:52 House.ibex.index.data
-rw-------. 1 user user  79899092 Jun 05 2012 14:19:41 Inbox
-rw-r--r--. 1 user user        13 Jun 05 2012 14:21:22 Inbox.cmeta
-rw-------. 1 user user    897024 Jun 05 2012 14:15:40 Inbox.ibex.index
-rw-------. 1 user user    851380 Jun 05 2012 14:15:40 Inbox.ibex.index.data
-rw-r--r--. 1 user user  1480522 Dec 20 2011 20:09:02 Jokes
-rw-r--r--. 1 user user        13 Dec 20 2011 20:09:04 Jokes.cmeta
-rw-------. 1 user user    23552 Dec 20 2011 20:09:04 Jokes.ibex.index
-rw-------. 1 user user      9616 Dec 20 2011 20:09:04 Jokes.ibex.index.data
-rw-r--r--. 1 user user    129746 Apr 26 2011 22:15:58 MISC
-rw-r--r--. 1 user user        13 Feb 17 2012 08:27:58 MISC.cmeta
-rw-------. 1 user user    12288 Apr 26 2011 22:19:05 MISC.ibex.index
-rw-------. 1 user user      3420 Apr 26 2011 22:19:05 MISC.ibex.index.data
drwx------. 2 user user      4096 Jun 09 2012 10:47:46 new
-rw-------. 1 user user        0 Jun 05 2012 14:17:27 Outbox
-rw-r--r--. 1 user user        13 Jun 05 2012 14:21:22 Outbox.cmeta
-rw-------. 1 user user      7168 Jun 05 2012 14:17:27 Outbox.ibex.index
-rw-------. 1 user user        8 Jun 05 2012 14:18:11 Outbox.ibex.index.data
-rw-r--r--. 1 user user        0 Jul 17 2009 01:49:41 PEOPLE
-rw-r--r--. 1 user user        13 May 23 2012 20:48:17 PEOPLE.cmeta
-rw-------. 1 user user      7168 Jul 17 2009 01:49:41 PEOPLE.ibex.index
-rw-------. 1 user user        8 Jul 17 2009 01:49:41 PEOPLE.ibex.index.data
-rw-r--r--. 1 user user  2111323 Jun 01 2011 21:44:59 PHOTO
-rw-r--r--. 1 user user        13 Jan 09 2012 12:55:22 PHOTO.cmeta
-rw-------. 1 user user    40960 Jun 01 2011 21:45:11 PHOTO.ibex.index
-rw-------. 1 user user    30908 Jun 01 2011 21:45:11 PHOTO.ibex.index.data
-rw-r--r--. 1 user user    173098 Dec 05 2011 15:54:55 politics
-rw-r--r--. 1 user user        13 Dec 05 2011 15:56:09 politics.cmeta
-rw-------. 1 user user    10240 Dec 05 2011 15:56:09 politics.ibex.index
-rw-------. 1 user user      1688 Dec 05 2011 15:56:09 politics.ibex.index.data
-rw-r--r--. 1 user user  8166767 May 02 2012 21:58:04 SCUBA
-rw-r--r--. 1 user user        13 May 02 2012 21:58:05 SCUBA.cmeta
-rw-------. 1 user user    19456 May 02 2012 21:58:05 SCUBA.ibex.index
-rw-------. 1 user user      8208 May 02 2012 21:58:05 SCUBA.ibex.index.data
-rw-r--r--. 1 user user    149934 Jul 18 2010 21:06:25 SEA
-rw-r--r--. 1 user user        39 Jul 18 2010 22:09:32 SEA.cmeta
-rw-------. 1 user user    18432 Jul 18 2010 21:07:08 SEA.ibex.index
-rw-------. 1 user user      7140 Jul 18 2010 21:07:08 SEA.ibex.index.data
-rw-r--r--. 1 user user 920749069 Jun 05 2012 14:17:27 Sent
-rw-r--r--. 1 user user        13 Jun 05 2012 14:21:22 Sent.cmeta
-rw-------. 1 user user    492544 Jun 05 2012 14:17:27 Sent.ibex.index
-rw-------. 1 user user    376012 Jun 05 2012 14:17:27 Sent.ibex.index.data
-rw-r--r--. 1 user user      1856 Jul 26 2009 20:18:42 SPACE
-rw-r--r--. 1 user user        13 Nov 06 2009 19:29:15 SPACE.cmeta
-rw-------. 1 user user      9216 Jul 26 2009 20:33:27 SPACE.ibex.index
-rw-------. 1 user user      284 Jul 26 2009 20:33:27 SPACE.ibex.index.data
-rw-r--r--. 1 user user        0 Dec 25 2009 14:03:27 SPAM_TRASH
-rw-r--r--. 1 user user        13 Apr 28 2012 16:22:25 SPAM_TRASH.cmeta
-rw-------. 1 user user      7168 Dec 25 2009 14:03:27 SPAM_TRASH.ibex.index
-rw-------. 1 user user        8 Dec 25 2009 14:03:27 SPAM_TRASH.ibex.index.data
-rw-r--r--. 1 user user        0 Jul 16 2009 09:02:09 Templates
-rw-r--r--. 1 user user        13 Jun 05 2012 14:21:22 Templates.cmeta
-rw-------. 1 user user      7168 Jul 17 2009 00:49:16 Templates.ibex.index
-rw-------. 1 user user        8 Jul 17 2009 00:54:46 Templates.ibex.index.data
drwx------. 2 user user      4096 Jun 09 2012 10:47:46 tmp
-rw-r--r--. 1 user user    16431 Oct 23 2011 22:46:23 trecks
-rw-r--r--. 1 user user        13 Oct 23 2011 22:46:33 trecks.cmeta
-rw-------. 1 user user      9216 Oct 23 2011 22:46:33 trecks.ibex.index
-rw-------. 1 user user        68 Oct 23 2011 22:46:33 trecks.ibex.index.data
-rw-r--r--. 1 user user  1227195 Apr 28 2012 16:24:16 WATER
-rw-r--r--. 1 user user        13 Apr 28 2012 16:25:42 WATER.cmeta
-rw-------. 1 user user    74752 Mar 18 2012 05:48:08 WATER.ibex.index
-rw-------. 1 user user    93560 Mar 18 2012 05:48:08 WATER.ibex.index.data
-rw-r--r--. 1 user user  4564480 May 31 2012 21:50:40 WORK
-rw-r--r--. 1 user user        13 May 31 2012 22:21:22 WORK.cmeta
-rw-------. 1 user user    119808 May 31 2012 22:21:22 WORK.ibex.index
-rw-------. 1 user user    123820 May 31 2012 22:21:22 WORK.iex.index.data

Looking at an old list of rpm packages I did on Feb 2011, I see
that the evolution version I had then was:

Code:

evolution-2.32.1-1.fc14.x86_64
evolution-data-server-2.32.1-1.fc14.x86_64
evolution-help-2.32.1-1.fc14.noarch
evolution-data-server-doc-2.32.1-1.fc14.noarch
evolution-perl-2.32.1-1.fc14.x86_64
evolution-data-server-devel-2.32.1-1.fc14.x86_64

and now it's:

Code:

evolution-data-server-devel-3.4.1-2.fc17.x86_64
evolution-NetworkManager-3.4.1-2.fc17.x86_64
evolution-3.4.1-2.fc17.x86_64
evolution-data-server-3.4.1-2.fc17.x86_64
evolution-perl-3.4.1-2.fc17.x86_64
evolution-help-3.4.1-2.fc17.noarch

Had there been any significant change between how
the evolution interprets its own indexing of database files
in version 2.32.1 and now in 3.4.1 ?

Has anyone encountered a similar problem and managed to solve it?

TIA for any help,
kaza.

kaza 06-11-2012 03:06 PM

Solved!

After some searching I found the site explaining the difference
between Evolution databases of version 2.32 and 3.x:

http://forums.linuxmint.com/viewtopic.php?f=42&t=88648

and how to translate from the old format to the new one.
The shell procedure explained there seemed too "manual"
to me to treat a database of tens of folders so I prefared
to write a Perl script that somewhat "automates" the
tree-structure of the mail folders. The script is far
from perfect (I had to deal with "Inbox" manually
because the script isn't aware of its special location in
destination) and far from efficient (copy-paste old template
and modify) but it did all the rest. Here it is, feel free to use/alter:

Code:

#!/usr/bin/perl -w

$|=1;        #        For unbuffered STDOUT.

$debug1 = 0;        #        Display src/dest directories paths.
$debug2 = 0;        #        Display each src directory *.cmeta file.
$debug3 = 0;
$debug4 = 0;        #        Display system commands before run.

$really_run_system = 1;        #        DBG

$rc_filename = "";
$src_dir = "";
$dest_dir = "";

$ls = "/usr/bin/ls";
$mb2md = "/usr/bin/mb2md";

undef(%src_files_to_dest_dirs);


if (@ARGV) {
        while (@ARGV) {
                if ($ARGV[0] eq "-rc") {
                        shift(@ARGV);
                        $rc_filename = shift(@ARGV);
                        &read_args_from_file($rc_filename);
                }        #        End of one searched arg.
                elsif ($ARGV[0] !~ m/^\-/) {
                        if ($src_dir eq "") {
                                $src_dir = shift(@ARGV);
                        }        #        End of first - src.
                        else {
                                $dest_dir = shift(@ARGV);
                        }        #        End of non first - dest.
                }        #        End of not a switch, assume directory name.
                else {
                        &err_exit("Unknown switch: \"$ARGV[0]\".");
                }        #        End of unknown switch.
        }        #        End of one arg.
}        #        End of arg(s) exist.
else {
        &display_usage();
        exit(1);
}        #        End of no args, display usage.

if ($src_dir eq "") {
        &err_exit("No source directory given.");
}        #        End of no scr dir given.

if ($dest_dir eq "") {
        &err_exit("No destination directory given.");
}        #        End of no scr dir given.

if (!(-d $src_dir)) {
        &err_exit("Source directory \"$src_dir\" not found.");
}        #        End of wrong src.

if (!(-d $dest_dir)) {
        &err_exit("Destination directory \"$dest_dir\" not found.");
}        #        End of wrong src.


if ($dest_dir !~ m/\/$/) {
        $dest_dir .= "/";
}        #        End of append slash if none.

&translate_evolution_mboxes_to_mdirs($src_dir,$dest_dir);
&perform_translation();

exit(0);







#################
sub display_usage
#################
{
print STDERR <<EOF;
Usage:
$0: [-rc <resource file name>] src_dir dest_dir




-rc <resource file name>:
        All arguments can be groupped in a resource file, a name of which is supplyed after "-rc".

src_dir:
        The root directory of the old Evolution mail files with quartuplets of files named like mail folders.

dest_dir:
        The destination directory where for each <folder_name> of src_dir a directory <.folder_name> is created.


EOF
}




#############
sub err_exit
#############
{
        my($message) = @_;

        print STDERR "$0: ERROR: $message\n";
        exit(1);
}        #        End of error exit.




###############
sub safe_system
###############
{
        my($sys_command) = @_;

        $debug4 && print STDERR "Running system command: \"$sys_command\"...\n";
        if ($really_run_system) {
                system("$sys_command");
                if ($?) {
                        &err_exit("Couldn't run \"$sys_command\".\nError code: ".$?.".");
                }        #        End of error code returned by system call.
                $debug4 && print STDERR "Finished running system command.\n";
        }        #        End of really run.
}        #        End of system call with error exit detect.








#######################
sub read_args_from_file
#######################
{
        my($args_filename) = @_;
        my($args_file);
        my($one_arg);

        if (-r $args_filename) {
                $args_file = `/bin/cat $args_filename`;
                if ($args_file =~ m/\-\w/) {
                        $args_file = "\n".$args_file."\n";
                        $args_file =~ s/\#.*\n//g;
                        $args_file =~ s/\n/ /g;
                        #$debug1 && print STDERR "ARGS FILE: \"$args_file\".\n";
                        while ($args_file =~ s/^\s*(\S+)\s+//) {
                                $one_arg = $1;
                                push(@ARGV,$one_arg);
                        }        #        End of get one arg from file.
                }        #        End of at least one argument found.
                else {
                        &err_exit("No arguments found in file \"$args_filename\".");
                }        #        End of nothing to do with args file.
        }        #        End of file is readable.
        else {
                &err_exit("Can't read file \"$args_filename\".");
        }        #        End of can not read.
}        #        End of read args from file.





#######################################
sub translate_evolution_mboxes_to_mdirs
#######################################
{
        my($src_dir,$dest_dir) = @_;
        my(%cmeta_files);
        my(%ibex_index_files);
        my(%ibex_index_data_files);
        my(%subdirs);
        my($sys_cmd);
        my($ls_out);
        my($ls_line);
        my($base_filename);
        my($cmeta_filename);
        my($ibex_index_filename);
        my($ibex_index_data_filename);
        my($sbd_dirname);
        my($base_foldername);
        my($dest_foldername);


        $debug1 && print STDERR "Src:  \"$src_dir\",\nDest: \"$dest_dir\".\n";

        $sys_cmd = $ls." -A1 ".$src_dir."/*.cmeta";
        $ls_out = `$sys_cmd`;

        while ($ls_out =~ s/(^[^\n\r]+)[\n\r]+//) {
                $ls_line = $1;
                $debug2 && print STDERR "*.cmeta ls line: \"$ls_line\".\n";
                if ($ls_line =~ m/(^\S+)\.cmeta$/) {
                        $base_filename = $1;
                        $cmeta_filename = $ls_line;
                }        #        End of non-empty.
                else {
                        &err_exit("Unexpected (not *.cmeta) file name in source dir: \"$src_dir\".\n");
                }        #        End of empty.

                $ibex_index_filename = $base_filename.".ibex.index";
                $ibex_index_data_filename = $base_filename.".ibex.index.data";
                $sbd_dirname = $base_filename.".sbd";

                if        ((-f $base_filename)                        &&
                        (-f $cmeta_filename)                        &&
                        (-f $ibex_index_filename)                &&
                        (-f $ibex_index_data_filename))        {
                        $base_foldername = $base_filename;
                        $base_foldername =~ s/^\S*\/([^\/]+$)/$1/;
                        $dest_foldername = $dest_dir.".".$base_foldername;

                        if (!(defined($src_files_to_dest_dirs{$base_filename}))) {
                                $src_files_to_dest_dirs{$base_filename} = $dest_foldername;
                        }        #        End of unique.
                        else {
                                &err_exit("\$src_files_to_dest_dirs{$base_filename} is redefined.");
                        }        #        End of redefined.

                        if (-d $sbd_dirname) {
                                &translate_evolution_mboxes_to_mdirs($sbd_dirname,$dest_foldername);
                        }        #        End of sub-directory found, need recursy.
                }        #        End of all parts seems to be present.
                else {
                        &err_exit("Some of the files of \"$base_filename\" are missing.");
                }        #        End of some parts are missing.
        }        #        End of get one line.
}        #        End of translate Evolution mail directory.





#######################
sub perform_translation
#######################
{
        my($src_filename);
        my($dest_fordername);
        my($sys_command);

        foreach $src_filename (sort keys %src_files_to_dest_dirs) {
                $dest_fordername = $src_files_to_dest_dirs{$src_filename};
                $sys_command = $mb2md." -s ".$src_filename." -d ".$dest_fordername;
                $debug3 && print STDERR "Sys command to perform: \"$sys_command\".\n";
                &safe_system($sys_command);
        }        #        End of one src.
}        #        End of perform the actual translation.

kaza


All times are GMT -5. The time now is 04:40 AM.