LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu
User Name
Password
Ubuntu This forum is for the discussion of Ubuntu Linux.

Notices


Reply
  Search this Thread
Old 08-28-2019, 04:28 PM   #1
JoLinuxUser
LQ Newbie
 
Registered: Aug 2019
Posts: 5

Rep: Reputation: Disabled
apt-offline fails to install updates to offline computer with ERROR: Failed to unlink /var/lib/apt/lists/partial/


I've been working to get apt-offline to update an offline ubuntu box running 16.04, but I've ran into some problems.

I believe software is installed correctly on the offline box. I run the following command with no errors:

Code:
sudo apt-offline set /var/tmp/apt-updates.sig
I then take the resulting .sig file to a Windows PC running apt offline and I'm able to pull down the updates.

If I then try to install the updates with the following command, I get ERRORs and the update never happens

I run:
Code:
sudo apt-offline install apt-offline.zip --verbose
I get the following:

Code:
VERBOSE: Namespace(allow_unauthenticated=False, func=<function installer at 0x7f591efb7b18>, install='apt-offline.zip', install_src_path=None, simulate=False, skip_bug_reports=False, verbose=True)
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_Release.gpg.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial_Release.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial_InRelease.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_Release.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-backports_Release.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.canonical.com_ubuntu_dists_xenial_Release.gpg.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.canonical.com_ubuntu_dists_xenial_Release.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_InRelease.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_InRelease.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.canonical.com_ubuntu_dists_xenial_InRelease.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial_Release.gpg.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_Release.gpg.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-backports_Release.gpg.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-backports_InRelease.
VERBOSE: Cleaning old update data file /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_Release.
VERBOSE: {}
VERBOSE: Great!!! No bugs found for all the packages that were downloaded.

VERBOSE: File is security.ubuntu.com_ubuntu_dists_xenial-security_Release.gpg, hence 'True'.
VERBOSE: security.ubuntu.com_ubuntu_dists_xenial-security_Release.gpg file synced to /var/lib/apt/lists/partial.
VERBOSE: security.ubuntu.com_ubuntu_dists_xenial-security_Release file synced to /var/lib/apt/lists/partial.
VERBOSE: security.ubuntu.com_ubuntu_dists_xenial-security_InRelease file synced to /var/lib/apt/lists/partial.
ERROR: Failed to unlink /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packagesapt-offline
ERROR: Failed to unlink /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_main_binary-i386_Packagesapt-offline
ERROR: Failed to unlink /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_main_i18n_Translation-enapt-offline
ERROR: Failed to unlink /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_main_dep11_Components-amd64.ymlapt-offline
This continues for every package in my apt-offline.zip file. When I view the contents of /var/lib/apt/lists/partial/ there is nothing there. I'm puzzled at why this is failing since everything seems correct up to this point. Does anyone have any ideas on why this is failing?
 
Old 08-30-2019, 05:49 AM   #2
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 10,875

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Have you tried the usual offenders? Permissions on all directories, disk space, known bug?
 
Old 08-30-2019, 09:30 AM   #3
JoLinuxUser
LQ Newbie
 
Registered: Aug 2019
Posts: 5

Original Poster
Rep: Reputation: Disabled
The issue seems related to permissions with /var/lib/apt/lists/partial/. I can't even do an ls on the directory without a sudo first.

I've traced the code down to the part were it is failing. It's failing when it attempts to decompress security.ubuntu.com_ubuntu_dists_xenial-security_main_binary-amd64_Packagesapt-offline.xz into /var/lib/apt/lists/partial.

I'm unable to put any files into this directory, even if I sudo first. looking at the permissions, it seems as _apt is the owner and the only one with Read/write/execute privileges. This box is brand new and there's been no work done on it, so I'm assuming it was configured this way by default. I'm not sure how to set the permissions so that root has access to read/write/execute in this directory. I think it will work if I can do that since I execute the entire apt-offline with a sudo. Any ideas?
 
Old 08-31-2019, 02:59 AM   #4
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 13,171
Blog Entries: 9

Rep: Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607
^ I think you may have found a bug (esp. since probably not too many people are using apt offline).
You should do your civic duty and report it (or see if it has been reported already and put your +1 there).

I think at some point in the not-too-distant past, apt changed something so it runs as an unprivileged user, not as root.

Be that as it may, show us
Code:
ls -al /var/lib/apt/lists/partial/
please.

- I get permission denied on my stretch box, so sudo might be necessary.
Let's play with it:
Code:
sudo ls -al /var/lib/apt/lists/partial/
sudo ls -al /var/lib/apt/lists/
sudo -u _apt ls -al /var/lib/apt/lists/partial/
 
Old 09-09-2019, 07:04 PM   #5
JoLinuxUser
LQ Newbie
 
Registered: Aug 2019
Posts: 5

Original Poster
Rep: Reputation: Disabled
Here is the contents of /var/lib/apt/lists:

Code:
total 1944
drwxr-xr-x 3 root root   4096 Aug 28 04:12 .
drwxr-xr-x 6 root root   4096 Aug 22 05:18 ..
-rw-rw-r-- 1 root root  11474 Aug 29 08:50 archive.canonical.com_ubuntu_dists_xenial_InRelease
-rw-rw-r-- 1 root root  10509 Aug 29 08:50 archive.canonical.com_ubuntu_dists_xenial_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:37 archive.canonical.com_ubuntu_dists_xenial_Release.gpg
-rw-rw-r-- 1 root root 106824 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-backports_InRelease
-rw-rw-r-- 1 root root 105859 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-backports_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:39 archive.ubuntu.com_ubuntu_dists_xenial-backports_Release.gpg
-rw-rw-r-- 1 root root 246846 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial_InRelease
-rw-rw-r-- 1 root root 245864 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial_Release
-rw-rw-r-- 1 root root    933 Aug 26 18:37 archive.ubuntu.com_ubuntu_dists_xenial_Release.gpg
-rw-rw-r-- 1 root root 109196 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-updates_InRelease
-rw-rw-r-- 1 root root 108231 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-updates_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:38 archive.ubuntu.com_ubuntu_dists_xenial-updates_Release.gpg
-rw-r----- 1 root root      0 Aug 29 08:50 lock
lrwxrwxrwx 1 root root      9 Aug  8 04:50 _Packages -> /Packages
drwx------ 2 _apt root   4096 Aug 29 08:50 partial
-rw-rw-r-- 1 root root 109198 Aug 29 08:50 security.ubuntu.com_ubuntu_dists_xenial-security_InRelease
-rw-rw-r-- 1 root root 108233 Aug 29 08:50 security.ubuntu.com_ubuntu_dists_xenial-security_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:37 security.ubuntu.com_ubuntu_dists_xenial-security_Release.gpg
-rw-r--r-- 1 root root 428669 Aug  8 04:50 Ubuntu%2016.04%20%5fXenial%5f%20-%20Build%20amd64%20LIVE%20Binary%2020160624-10:47_dists_xenial_main_binary-amd64_Packages
-rw-r--r-- 1 root root 340805 Aug  8 04:50 Ubuntu%2016.04%20%5fXenial%5f%20-%20Build%20amd64%20LIVE%20Binary%2020160624-10:47_dists_xenial_main_binary-i386_Packages
Notice _apt owns the directory partial which I believe is the issue. I can list partial if I do:
Code:
sudo ls -la /var/lib/apt/lists/partial
Code:
total 1184
drwx------ 2 _apt root   4096 Aug 29 08:50 .
drwxr-xr-x 3 root root   4096 Aug 28 04:12 ..
-rw-rw-r-- 1 root root  11474 Aug 29 08:50 archive.canonical.com_ubuntu_dists_xenial_InRelease
-rw-rw-r-- 1 root root  10509 Aug 29 08:50 archive.canonical.com_ubuntu_dists_xenial_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:37 archive.canonical.com_ubuntu_dists_xenial_Release.gpg
-rw-rw-r-- 1 root root 106824 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-backports_InRelease
-rw-rw-r-- 1 root root 105859 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-backports_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:39 archive.ubuntu.com_ubuntu_dists_xenial-backports_Release.gpg
-rw-rw-r-- 1 root root 246846 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial_InRelease
-rw-rw-r-- 1 root root 245864 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial_Release
-rw-rw-r-- 1 root root    933 Aug 26 18:37 archive.ubuntu.com_ubuntu_dists_xenial_Release.gpg
-rw-rw-r-- 1 root root 109196 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-updates_InRelease
-rw-rw-r-- 1 root root 108231 Aug 29 08:50 archive.ubuntu.com_ubuntu_dists_xenial-updates_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:38 archive.ubuntu.com_ubuntu_dists_xenial-updates_Release.gpg
-rw-rw-r-- 1 root root 109198 Aug 29 08:50 security.ubuntu.com_ubuntu_dists_xenial-security_InRelease
-rw-rw-r-- 1 root root 108233 Aug 29 08:50 security.ubuntu.com_ubuntu_dists_xenial-security_Release
-rw-rw-r-- 1 root root    916 Aug 26 18:37 security.ubuntu.com_ubuntu_dists_xenial-security_Release.gpg
The issue seems to come in when apt-offline tries to uncompress the package into the partial directory. Is there a way to change the ownership here so that apt-offline can write to the partial directory?
 
Old 09-10-2019, 02:34 AM   #6
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 13,171
Blog Entries: 9

Rep: Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607
Quote:
Originally Posted by JoLinuxUser View Post
Is there a way to change the ownership here
Sure.
Code:
chown -R user:group /var/lib/apt/partial
where you substitute user and group for the user and group you want.
And no, I don't know what user and group that should be.
TBH, I don't even know if that's the issue; your error message from post #1 does not explicitely say "permission denied" anywhere, although that does seem the likely explanation.
I repeat: I have the same permissions & ownerships here, and everything works.
I think you need to turn to the devs of apt-offline.

Or...
Quote:
Originally Posted by ondoho View Post
^ I think you may have found a bug (esp. since probably not too many people are using apt offline).
You should do your civic duty and report it (or see if it has been reported already and put your +1 there).
 
Old 09-11-2019, 08:01 PM   #7
JoLinuxUser
LQ Newbie
 
Registered: Aug 2019
Posts: 5

Original Poster
Rep: Reputation: Disabled
I'm posting this here for others who may encounter this. I finally found out what the issue was. It was rather annoying actually, and somewhat difficult to trace down. The culprit turned out to be a missing dependency in python necessary for apt-online to work. Python-lzma is required; however, there is no indication given to the user when it is not installed. This bit of code in AptOfflineCoreLib.py calls a function in APTOfflineLib.py:
Code:
   def magic_check_and_uncompress(self, archive_file=None, filename=None):
                
            if MagicLib is False:
                    log.err("Please ensure libmagic is installed\n")
                    return False

            magicMIME = AptOfflineMagicLib.open(AptOfflineMagicLib.MAGIC_MIME_TYPE)
            magicMIME.load()
            
            retval = False
            if magicMIME.file( archive_file ) == "application/x-bzip2" or magicMIME.file( archive_file ) == "application/gzip" or magicMIME.file(archive_file) == "application/x-xz":
                    temp_filename = os.path.join(self.apt_update_target_path, filename + app_name)
                    filename = os.path.join(self.apt_update_target_path, filename)
                    if magicMIME.file( archive_file ) == "application/x-bzip2":
                        retval = self.decompress_the_file( archive_file, temp_filename, "bzip2" )
                    elif magicMIME.file( archive_file ) == "application/gzip":
                        retval = self.decompress_the_file( archive_file, temp_filename, "gzip" )
                    elif magicMIME.file(archive_file) == "application/x-xz":
            ====>       retval = self.decompress_the_file(archive_file, temp_filename, "xz")
                    else:
                        log.verbose("No filetype match for %s\n" % (filename) )
                        retval = False
It's supposed to return TRUE on success. A closer look at APTOfflineLib.py reveals this:
Code:
modLZMA = True
try:
        import lzma
except ImportError:
        modLZMA = False
which returns FALSE if lzma is not installed (or TRUE if otherwise). Then there's this:
Code:
def decompress_the_file( self, archive_file, target_file, archive_type ):
                '''Extracts all the files from a single condensed archive file'''
                if archive_type == "bzip2" or archive_type == "gzip" or archive_type == "xz":
                        if archive_type == "bzip2":
                                try:
                                        read_from = bz2.BZ2File( archive_file, 'r' )
                                except IOError:
                                        return False
                        elif archive_type == "gzip":
                                try:
                                        read_from = gzip.GzipFile( archive_file, 'r' )
                                except IOError:
                                        return False
                        elif archive_type == "xz":
                                if modLZMA is True:
                                        try:
                                                read_from = lzma.LZMAFile(archive_file, 'rb')
                                        except IOError:
                                                return False //but if we fail, oh well just return FALSE
                                else:
                                        return False
                        else:
                                return False
                                    
                        try:
                                write_to = open ( target_file, 'wb' )
                        except IOError:
                                return False
                        
                        if self.TarGzipBZ2_Uncompress( read_from, write_to ) != True:
                                #INFO: Return False for the stream that failed.
                                return False
                        write_to.close()
                        read_from.close()
                        return True
Which returns FALSE if python-lzma is not installed. Doesn't tell you that it isn't installed, it just fails. I post this so that others may look at this. The lesson learned is to double-check you dependencies!
 
1 members found this post helpful.
Old 09-11-2019, 09:11 PM   #8
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782
it is there as a recommends

( this is on Sid )
Code:
apt show apt-offline
Package: apt-offline
<snip>
Depends: python3:any (>= 3.3.2-2~), apt, less, python3-magic
Recommends: python-soappy, debian-archive-keyring, python-lzma, python3-debianbts
You could report it as a dependency to the package maintainer
But I've seen Debian maintainers argue that since --install-recommends is the Debian default those who disable that are on there own (or words to that effect).
 
Old 09-12-2019, 01:07 AM   #9
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 13,171
Blog Entries: 9

Rep: Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607Reputation: 3607
Please mark your thread SOLVED.
Others will benefit.
 
Old 09-12-2019, 09:11 AM   #10
JoLinuxUser
LQ Newbie
 
Registered: Aug 2019
Posts: 5

Original Poster
Rep: Reputation: Disabled
Firerat you are correct. python-lzma is listed as a dependency, but what makes this difficult is that the offline box is not connected to the internet. I bet this would work great if the box was connected to the internet, but this is the whole point of apt-offline. I was finally able to make .sig files and get the package lists updated, but have yet to try to actually install something new. That is the next (and hopefully final) step. I will mark this thread as solved. Thanks for all those who offered up help.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
When I run apt-update, I get a /var/lib/apt/partial error kb2tfa Debian 1 10-24-2017 03:43 PM
/var/lib/apt/list/partial eating all of the disk space. replica9000 Debian 1 06-16-2014 04:47 PM
[SOLVED] 404errors when running apt-get update already tried sudo rm /var/lib/apt/lists/* -vf rasec8711 Linux - Software 2 05-31-2012 11:33 AM
LXer: Unique Sorting Of Lists And Lists Of Lists With Perl For Linux Or Unix LXer Syndicated Linux News 0 09-05-2008 02:50 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu

All times are GMT -5. The time now is 01:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration