[SOLVED] How to provide non-free firmware files to the Debian Jessie Installer.

luvr · 05-15-2015, 10:00 AM

When I attempted to install Debian 8 (jessie) on my laptop, I could not get a network connection, because the installer told me that it was missing the required firmware files for my network interfaces:

Code:

Some of your hardware needs non-free firmware files to operate.  The firmware can be loaded from
removable media, such as a USB stick or floppy.

The missing firmware files are: iwlwifi-7260-9.ucode iwlwifi-7260-8.ucode

If you have such media available now, insert it, and continue.
Load missing firmware from removable media?

Initially, I assumed that this was just a minor issue: Instead of the wireless interface, I could use wired ethernet to get a network connection, and get the wireless interface in an operational state after installation was completed.

However, when I continued, I got another, similar message about another firmware file:

Code:

The missing firmware files are: rtl_nic/rtl8411-2.fw

That was a bigger problem: Even the wired ethernet interface required firmware files that the installer could not find.

I tried to provide the required firmware files to the installer, but it kept complaining that it could not find them. It seems to me that the exact procedure to provide the firmware files to the installer, is not particularly well documented (at least, not at a place where I could find it), so quite a bit of experimentation was required for me to overcome this issue. In this post, I will explain what worked for me.

First, of course, I had to actually find the required firmware files. To that end:

I went to the Debian site.
I clicked the Debian Packages link, to continue to the Debian Packages page.
Under the Packages header, I selected the Search the contents of packages link.
In the Keyword: field, I entered: iwlwifi.
Under Display:, I selected the option labelled packages that contain files whose names contain the keyword.
I left the Distribution: value unchanged at stable.
I changed the Architecture: value to 64-bit PC (amd64).
I clicked the Search button.
In the list of search results that appeared next, I looked for the two files that I needed: iwlwifi-7260-8.ucode and iwlwifi-7260-9.ucode. Both files were present in the firmware-iwlwifi package.
I clicked one of the firmware-iwlwifi links, to move on to the Binary firmware for Intel Wireless cards page.
Under the Download firmware-iwlwifi header, I clicked the all architecture link to continue.
On the next page, I clicked the link to the site from which I wanted to download the file.

The Debian package firmware-iwlwifi_0.43_all.deb got downloaded for me.

I repeated the above procedure, to search the contents of packages by keyword rtl8411.

As a result, package firmware-realtek_0.43_all.deb was downloaded.

Next, it was time to extract the contents of the downloaded packages:

I opened a command-line terminal window, and went to the directory where the packages were downloaded―which, in my case, was my personal download directory:
Code:
```
cd ~/Downloads
```

I extracted the contents of both packages into the current directory:
Code:
```
dpkg-deb --extract firmware-iwlwifi_0.43_all.deb .
dpkg-deb --extract firmware-realtek_0.43_all.deb .
```
This step created two new subdirectories: lib and usr.

The lib directory, in turn, contained a further subdirectory, firmware, that held all firmware files extracted from both packages―including, but not limited to, the files that I (or, rather, the Debian installer) was looking for.

I copied the entire contents of the firmware subdirectory to a USB stick that was formatted with a FAT filesystem.

The directory listing of the USB stick, then, looked like this:

Code:

total 13424
drwx------ 2 luvr luvr  16384 May 15 13:58 intel
-rw-r--r-- 1 luvr luvr 337520 Jun 16  2014 iwlwifi-1000-5.ucode
-rw-r--r-- 1 luvr luvr 337572 Jun 16  2014 iwlwifi-100-5.ucode
-rw-r--r-- 1 luvr luvr 689680 Jun 16  2014 iwlwifi-105-6.ucode
-rw-r--r-- 1 luvr luvr 701228 Jun 16  2014 iwlwifi-135-6.ucode
-rw-r--r-- 1 luvr luvr 695876 Jun 16  2014 iwlwifi-2000-6.ucode
-rw-r--r-- 1 luvr luvr 707392 Jun 16  2014 iwlwifi-2030-6.ucode
-rw-r--r-- 1 luvr luvr 670484 Jun 16  2014 iwlwifi-3160-7.ucode
-rw-r--r-- 1 luvr luvr 667284 Jun 16  2014 iwlwifi-3160-8.ucode
-rw-r--r-- 1 luvr luvr 666792 Jun 16  2014 iwlwifi-3160-9.ucode
-rw-r--r-- 1 luvr luvr 150100 Jun 16  2014 iwlwifi-3945-2.ucode
-rw-r--r-- 1 luvr luvr 187972 Jun 16  2014 iwlwifi-4965-2.ucode
-rw-r--r-- 1 luvr luvr 353240 Jun 16  2014 iwlwifi-5000-2.ucode
-rw-r--r-- 1 luvr luvr 340696 Jun 16  2014 iwlwifi-5000-5.ucode
-rw-r--r-- 1 luvr luvr 337400 Jun 16  2014 iwlwifi-5150-2.ucode
-rw-r--r-- 1 luvr luvr 454608 Jun 16  2014 iwlwifi-6000-4.ucode
-rw-r--r-- 1 luvr luvr 444128 Jun 16  2014 iwlwifi-6000g2a-5.ucode
-rw-r--r-- 1 luvr luvr 677296 Jun 16  2014 iwlwifi-6000g2a-6.ucode
-rw-r--r-- 1 luvr luvr 679436 Jun 16  2014 iwlwifi-6000g2b-6.ucode
-rw-r--r-- 1 luvr luvr 463692 Jun 16  2014 iwlwifi-6050-4.ucode
-rw-r--r-- 1 luvr luvr 469780 Jun 16  2014 iwlwifi-6050-5.ucode
-rw-r--r-- 1 luvr luvr 683236 Jun 16  2014 iwlwifi-7260-7.ucode
-rw-r--r-- 1 luvr luvr 679780 Jun 16  2014 iwlwifi-7260-8.ucode
-rw-r--r-- 1 luvr luvr 679380 Jun 16  2014 iwlwifi-7260-9.ucode
-rw-r--r-- 1 luvr luvr 690452 Jun 16  2014 iwlwifi-7265-8.ucode
-rw-r--r-- 1 luvr luvr 691960 Jun 16  2014 iwlwifi-7265-9.ucode
drwx------ 2 luvr luvr  16384 May 15 13:58 RTL8192E
drwx------ 2 luvr luvr  16384 May 15 13:58 RTL8192SU
drwx------ 2 luvr luvr  16384 May 15 13:58 rtl_nic
drwx------ 2 luvr luvr  16384 May 15 13:58 rtlwifi

Note, in particular, that the USB stick included the contents of the firmware directory, not the directory itself!

Finally, the time had come to retry the Debian installation. I simply booted the computer off the Debian 8.0.0 medium, and when it complained again about the missing firmware files, I took the following steps:

I inserted the USB stick.
I pressed the <CTRL>+<Alt>+<F2> key combination, to switch to the second virtual terminal.
I ran the blkid command, to find out under which device name the USB stick was known: /dev/sdc. Its FAT filesystem, which contained the firmware files, then, was /dev/sdc1.
I mounted the FAT filesystem under the /lib/firmware mountpoint―which, however, I had to create first:
Code:
```
mkdir /lib/firmware
mount /dev/sdc1 /lib/firmware
```

I pressed <CTRL>+<Alt>+<F5> to continue the installation process.
The firmware files were successfully loaded, and I got prompted for my wireless networking parameters.

The remainder of the installation went flawlessly.

beachboy2 · 05-15-2015, 10:23 AM

luvr,

That is very useful information.

Might it be good idea to produce this as a blog for others to refer to?

273 · 05-15-2015, 02:32 PM

Yes, thanks, I've been through that same process so I know it could help others.
I do, however, seem to recall that my wired network card worked without the non-free firmware despite the complaint -- did you try to install regardless?
Mine was on a desktop PC so I was doubly surprised when the network card was said to need firmware!

luvr · 05-15-2015, 02:53 PM

Quote:

Originally Posted by 273

I do, however, seem to recall that my wired network card worked without the non-free firmware despite the complaint -- did you try to install regardless?

Yes, I did try to continue, since I was surprised that even the ethernet interface would require non-free firmware. It didn't work—no network hardware was available. Had it worked, then I don’t think I would have gone through the trouble of figuring out how to load the firmware during installation.

273 · 05-15-2015, 02:59 PM

Quote:

Originally Posted by luvr

Yes, I did try to continue, since I was surprised that even the ethernet interface would require non-free firmware. It didn't work—no network hardware was available. Had it worked, then I don’t think I would have gone through the trouble of figuring out how to load the firmware during installation.

Good to know, thanks, I was doing other things during my install and really did not recall exactly what I did.

luvr · 05-15-2015, 03:10 PM

Quote:

Originally Posted by beachboy2

Might it be good idea to produce this as a blog for others to refer to?

Done: I reposted it as an LQ Blogs entry.

Head_on_a_Stick · 05-15-2015, 04:23 PM

@OP: Have you tried one of the unofficial ISO images with the non-free firmware already included?
http://cdimage.debian.org/cdimage/un...ding-firmware/

Seems a bit simpler to me...

luvr · 05-16-2015, 03:13 AM

Quote:

Originally Posted by Head_on_a_Stick

Have you tried one of the unofficial ISO images with the non-free firmware already included?
Seems a bit simpler to me...

True—That would have been significantly easier.
However, I got intrigued by the issue, enough so to decide that I wouldn’t simply give in—at least not without a fight.

jlinkels · 05-16-2015, 06:47 AM

I have been more than once in the same situation -- and failed. So thanks for your useful post.

OT: I discover now I cannot mark your post as useful because it is the first one in the thread.

jlinkels

ardvark71 · 05-16-2015, 07:05 AM

Quote:

Originally Posted by beachboy2

That is very useful information.

+1

Thanks to the OP for posting it.

Regards...

luvr · 10-14-2015, 08:26 AM

Some time ago, I upgraded Debian 8 (“jessie”) on my laptop to Debian Testing (cfr. my sources.list). At the time, Debian Testing was running Linux Kernel 4.1, and whenever I booted Debian, the following messages appeared:

Code:

iwlwifi 0000:02:00.0: firmware: failed to load iwlwifi-7260-13.ucode (-2)
iwlwifi 0000:02:00.0: firmware: failed to load iwlwifi-7260-12.ucode (-2)
iwlwifi 0000:02:00.0: firmware: failed to load iwlwifi-7260-11.ucode (-2)
iwlwifi 0000:02:00.0: Firmware has old API version, expected v12 through v13, got v10.
iwlwifi 0000:02:00.0: New firmware can be obtained from http://www.intellinuxwireless.org/.

Apparently, the iwlwifi module under Linux Kernel 4.1 went looking for a “iwlwifi-7260-13.ucode” firmware blob, which wasn’t available on my system. Fortunately, however, it would fall back to the earlier “iwlwifi-7260-10.ucode” file, which was provided by the “firmware-iwlwifi” package, and, as a result, my wireless networking interface remained operational after all.

Even so, I decided to download the new firmware as suggested by the text, and copied it to the “/lib/firmware” directory. That got rid of the warning messages.

Recently, Debian Testing was upgraded to Linux Kernel 4.2, and that expected an even newer firmware file, as evidenced by the following messages:

Code:

iwlwifi 0000:02:00.0: firmware: failed to load iwlwifi-7260-15.ucode (-2)
iwlwifi 0000:02:00.0: Direct firmware load for iwlwifi-7260-15.ucode failed with error -2
iwlwifi 0000:02:00.0: firmware: direct-loading firmware iwlwifi-7260-14.ucode
iwlwifi 0000:02:00.0: loaded firmware version 25.30.14.0 op_mode iwlmvm

I could, of course, have ignored these messages, since the absence of the “iwlwifi-7260-15.ucode” file didn’t prevent my wireless networking interface from working. Alternatively, I could manually download the new file—but when I tried that, I found that the file wasn’t yet listed as a production-ready firmware version.

Since the firmware file that I was looking for, wasn’t available yet, I decided to try and cobble up a bash script to automate the download and install process (I’m a strong proponent of automating repetitive tasks), so I could simply run the script every now and then, to verify if the new firmware file had already arrived. The result, i.e., the “get-iwlwifi-firmware” script, which I attached to this post, takes the following steps:

It creates a directory, “opt/firmware-iwlwifi”, under your home location. The firmware files will be downloaded into this newly created directory.
It gets the web page that lists the available firmware files, and extracts the names of the “iwlwifi-7260” archive files from it. It subsequently downloads all of these archive files.
After the download completes, it looks for any archives that have a “.1” filename extension. It assumes that such files are new downloads of files that were already downloaded on an earlier occasion.
If the newly downloaded copy is identical to the earlier one (i.e., the one without the “.1” extension), then the new copy is simply deleted.
If, on the other hand, the new copy is different from the earlier one, then the old one is deleted (along with the contents that was extracted from it), and the new one is renamed to take its place.
Each of the downloaded archive files is then processed. If the contents of an archive has not yet been extracted, then it must be new; its contents is then extracted, and the extracted firmware blob is installed to the “/lib/firmware” directory.

Even though the script is not particularly fail-safe (one of the most obvious issues against which it doesn’t guard, is the presence of a stray “.tgz.1” file when it is run), it works well enough under normal circumstances. Feel free to use it, or to adapt it to your own needs, if you consider it useful.

cynwulf · 10-14-2015, 11:22 AM

Quote:

Originally Posted by Head_on_a_Stick

@OP: Have you tried one of the unofficial ISO images with the non-free firmware already included?
http://cdimage.debian.org/cdimage/un...ding-firmware/

Seems a bit simpler to me...

It's simpler in hindsight, but not to those who downloaded an official images and are then faced with downloading again on reading this kind of advice. Debian don't really advertise those, so it's not really that helpful to just keep posting links to unofficial images whenever this kind of thing crops up (as it always has).

The situation regarding firmware during the install was never documented very clearly, but the Debian wiki has always covered it (since squeeze was testing). This is because DFSG regards Linux kernel firmware as closed source blobs (a la FSF) and won't redistribute it as part of the 'main' distribution without accompanying source (of which there is none). So in other words it doesn't get discussed much - it's up to the user to find it and install it (yet the whole approach to non-free in debian has slackened of late - and last time I checked, you do get offered the chance to enable those source and contrib sources at install time).

So in view of all that, reading the relevant Debian wiki page isn't a bad idea:

https://wiki.debian.org/Firmware#Fir...e_installation

The second paragraph and link to the firmware tarball should be relevant. Usually extracting the tarball to a directory called /firmware on the flash stick will work.

The third paragraph suggests manually mounting the flash drive to /media , if it's not mounted automatically by the installer. check-missing-firmware.sh should look for it there.

The mailing list link on the aforementioned wiki page seems to suggest that the .deb in the / of the flash memory stick can also work - maybe it does, can't remember.

jkwilborn · 11-18-2015, 07:34 PM

I have multiple computers that require different types of firmware blobs. What I've found that works with Debian systems is to put all of your firmware on a USB stick, then when the installer enters the detecting network stage and complains requesting firmware, you back out to a shell. In the shell you need to mount your usb stick by making a /firmware directory under the root and mounting the stick there. When you go back to detecting network and he finds it in the /firmware directory, under the root. This is actually documented in the boot or installation setup about how it searches for these files. It specifically looks into a /firmware under the root for the blobs.

My particular stick has the normal files and some directories as suggested by the error message, such as an rtl_nic/ directory for those firmware blobs. I do believe that they should make some kind of mechanism for new users to have available to at least get them on-line. One of the attractions of the other flavours load the non-free blobs. At least let you stick in a USB stick and use that without knowing about making directories and mounting to a file system. I have yet to have one that would release the DVD drive to allow for changing it to the next DVD. A few items to change... Such is life..

As for other Linux variations, try a bunch of them via the 'live' type DVD, so you can run it and see if you like it. Most also give you the option of installation. Live DVD's with the installation option may be the way ahead for installations.

Best of luck

Jack

luvr · 11-20-2015, 08:11 AM

Quote:

Originally Posted by cynwulf

The second paragraph and link to the firmware tarball should be relevant. Usually extracting the tarball to a directory called /firmware on the flash stick will work.

That's actually one of the things that I had tried, but it didn't work for me. It did eventually help me come up with the idea to copy the firmware files (which I extracted from the ".deb" packages) to the root directory of the stick, and mount it under "/firmware".

Quote:

The third paragraph suggests manually mounting the flash drive to /media , if it's not mounted automatically by the installer. check-missing-firmware.sh should look for it there.

That's another thing that I had tried, and that also was unsuccesful for me.

Quote:

The mailing list link on the aforementioned wiki page seems to suggest that the .deb in the / of the flash memory stick can also work - maybe it does, can't remember.

Again, something that I had tried, without success.

Perhaps these things should have worked, and something went wrong on my side (I'm not excluding a bad case of PEBKAC or PICNIC either...); I just don't know.

whodatder · 06-17-2016, 10:29 AM

thank you sir, you saved me a lot of time and effort!