LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 02-16-2018, 10:24 AM   #1
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
How to bisect a kernel without downloading the entire tree


When I google this, I get a load of HOWTOs that mostly start with cloning the whole kernel tree from ca. version 2.6. By now this amounts to about 3 GB, which is a huge waste of bandwidth and not consistent with my broadband package (I have an economy package with a monthly limit of 3GB and most of the time that is more than ample for my needs).

I only need a small twig of the tree for my tests. I know that 4.13.16 (the last in the 4.13 series) works for me and that 4.14.0 does not. I would like to clone that part of the tree only and then test the individual commits between those two versions. Can anyone give me a git command for doing that?
 
Old 02-17-2018, 03:10 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,899

Rep: Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318
do you mean something like this: https://stackoverflow.com/questions/...he-entire-tree
 
1 members found this post helpful.
Old 02-17-2018, 07:36 AM   #3
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
Yes, that looks like what I want. Thank you.
 
Old 02-18-2018, 10:15 AM   #4
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
I've unsolved this one again because it turns out that this doesn't actually solve my problem. Evidently what I think of as a "branch" and what git calls a branch are two very different things! Even with -b v4.13 --single-branch, the amount that needs to be downloaded is way too big. I can see this by looking at how much has already come over at the 1% or 2% mark; the total is not going to be much less than before.

Let me be a bit more specific. I know that the latest 4.13 kernel (4.13.16) will boot normally for me. I have also discovered that no 4.14 kernel will do so (unless I deactivate acpi) and none of the 4.14 release candidates from Linus's tree either. So my last good kernel is 4.13.16 and my first bad one is 4.14-rc1. This is where I need to bisect. I have the time to do it and I have the space; I've just created an empty 20 GB partition where I can build as many parallel kernels as I like. What I don't have so far is a way to get at all the intervening changes, given that I have a 3 GB per month download limit. If I had a local friend who used Linux and had an all-you-can-eat tariff, I could just ask him to clone the whole tree on his machine and put it on a memory stick. But there is no one who matches up.

Having this limit never bothered me before because I'm not interested in watching YouTube or iPlayer, or streaming music. Most of my downloads are software updates, a few MB here and there. And as a pensioner with a restricted income, I have gone for the most affordable option, not the snazziest. But now I have a problem!
 
Old 02-18-2018, 10:37 AM   #5
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,899

Rep: Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318
https://git.kernel.org/pub/scm/linux....git/refs/tags
you can find a kernel you need and download it. the size is at about 150MB I think.
Also somehow you can download diff (between two tags), but need to check the correct way to do that.
 
Old 02-18-2018, 11:47 AM   #6
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
That's the site I got the 4.14 release candidates from. But I can't see any way to get the intermediate versions by hand. I mean how would I even recognise them? Each commit comes with a signature, but I can't see how those relate to the version numbers.

If I knew for certain that one particular commit was the next one after the 4.13.16 release, that would be something. The releases don't even seem to come out in chronological order. For example, 4.13.16 is dated 24th November on the main kernel.org site, but 4.14 is 12th November!

Last edited by hazel; 02-18-2018 at 11:53 AM.
 
Old 02-21-2018, 07:31 PM   #7
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
You can goto https://www.kernel.org/pub/linux/kernel/v4.x/ and download the patch files you need, You can probably just search the patches for the changes you're looking for.
 
Old 02-22-2018, 01:46 AM   #8
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
Quote:
Originally Posted by AwesomeMachine View Post
You can goto https://www.kernel.org/pub/linux/kernel/v4.x/ and download the patch files you need, You can probably just search the patches for the changes you're looking for.
No, that won't work. Please read through my posts again and you will see that the release candidate for the 4.14 series already shows the problem while the latest patched version of 4.13 does not. It is therefore the intermediate unpublished part of the tree that needs to be searched.

Fortunately I have discovered that my computer virgin friend (the only Linux user I know around here) has a monthly usage limit of 12 GB and she is happy to let me have a one-off gift of 3 GB for my project. I suggested jocularly that it would be my fee for teaching her how to use a computer. So I shall install git on her machine, clone the kernel git tree onto a memory stick and carry it home in triumph (absit omen) to copy to my machine.

I'll report on how it went.
 
Old 02-22-2018, 05:06 AM   #9
Shadow_7
Senior Member
 
Registered: Feb 2003
Distribution: debian
Posts: 4,137
Blog Entries: 1

Rep: Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874
The full tarballs are like 100MB from kernel.org. Which you would only need once plus a couple patch files for the version you want and the next release.

https://www.kernel.org/pub/linux/kernel/v4.x/

... 4.13.16:
https://www.kernel.org/pub/linux/ker...4.13.16.tar.xz

... 4.14.0
https://www.kernel.org/pub/linux/ker...ux-4.14.tar.xz

The patch from 4.13 to 4.14 is only 8.5MB. The patch from 4.13 to 4.13.16 is 2.7MB. The tarball for 4.13.0 is 100.6MB.

https://www.kernel.org/pub/linux/ker...4.13.16.tar.xz
https://www.kernel.org/pub/linux/ker...tch-4.13.16.xz
https://www.kernel.org/pub/linux/ker.../patch-4.14.xz

So less than 120MB of downloads going that route. The diff you're looking for is likely IN the 4.14 patch file, to shorten your search of sorts. Or even less going with incremental patches. Although I'm not finding one to cross 4.13.16 to 4.14.0.

https://cdn.kernel.org/pub/linux/kernel/v4.x/incr/

Last edited by Shadow_7; 02-22-2018 at 05:23 AM. Reason: the .gz one is 50%+ bigger.
 
Old 02-22-2018, 05:31 AM   #10
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
Actually the published patches are collections of individual commits that have stood the test of time. If you complain to the kernel mailing list that you have a problem, they will ask you to use git bisect to find out the individual commit that caused it.

A single kernel gzipped tarball is about 150 MB in my experience (less for published versions because they provide xzipped tarballs too), but the git repository duplicates the entire tree for each commit and there are several each day. That makes it easier for developers to go back to an earlier good version if things don't pan out, but more laborious for the likes of you and me to tap into.

If you have once cloned the tree, you can keep it up to date quite economically by just pulling down the new stuff, but there are no shortcuts for the initial clone. I should know! I've been googling for days trying to find a way around this.
 
Old 02-22-2018, 05:45 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,899

Rep: Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318Reputation: 7318
probably a shallow clone can help you.
 
Old 02-22-2018, 06:06 AM   #12
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 7,597

Original Poster
Blog Entries: 19

Rep: Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455Reputation: 4455
Quote:
Originally Posted by pan64 View Post
probably a shallow clone can help you.
Yup, good guess! I tried that too. What I actually did was to try various arguments that looked like they might cut down on bandwidth, including cloning a branch and doing a shallow clone (and then both at once!). I let it download until it showed 2%, then interrupted it and calculated what the total would be. It was reduced by using these options but not by much. I'd still be over the limit.

Now some people here will say that it's entirely my fault for being such a skinflint and choosing a broadband contract with a low usage ceiling. But this is the first time since I've been online (about 15 years I think) that I have ever run into a problem like this. Normally what I have paid for is more than ample for my needs so why should I pay more? No, I think for this one-off problem, getting help from a friend with a bigger allowance is the way to go.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] git bisect the kernel, confused with version tags lordadamson Linux - Newbie 1 12-05-2014 11:21 AM
Is it safe to do 'git bisect' several time on kernel sam-the-6 Linux - Kernel 6 08-03-2014 04:59 PM
Copy an entire folder tree pshankland Linux - Newbie 3 09-28-2010 03:48 AM
ls entire subdirectory tree snocked Linux - General 6 07-27-2006 08:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 07:08 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration