LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud
User Name
Password
Linux - Virtualization and Cloud This forum is for the discussion of all topics relating to Linux Virtualization and Linux Cloud platforms. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. OpenStack, CloudStack, ownCloud, Cloud Foundry, Eucalyptus, Nimbus, OpenNebula and all other Linux Cloud platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.

Notices


Reply
  Search this Thread
Old 02-26-2011, 11:18 AM   #1
tkmsr
Member
 
Registered: Oct 2006
Distribution: Ubuntu,Open Suse,Debian,Mac OS X
Posts: 798

Rep: Reputation: 39
how to reduce time of git pulling each time when you do a make world on Xen sourc


I am compiling xen from source and each time I do a make world
it basically gives some or the other error
my problem are not those errors ( I am trying to debug them)
but the problem is each time when I do a make world

Xen basically pulls things from git repository
Code:
   + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
   + mkdir linux-2.6-pvops.git.tmp
   + rmdir linux-2.6-pvops.git.tmp
   + git clone -o xen -n
git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
linux-2.6-pvops.git.tmp
   Initialized empty Git repository in
/usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
   remote: Counting objects: 1941611, done.
   remote: Compressing objects: 100% (319127/319127), done.
remote: Total 1941611 (delta 1614302), reused 1930655 (delta 1604595)
**Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s, done.**
and if you notice the last line it is still consuming my bandwidth
pulling things from internet.How can I stop this step each time and
use existing git repository?
When I searched an answer to this question I came across a link
http://lists.xensource.com/archives/.../msg00476.html
http://web.archiveorange.com/archive...CfIzdpcmkWRz5w

but I am not able to clearly understand where Boris mentions

Quote:
make xen etc.
"Then clone JF's Git Repo, checkout git branch as you want and build
corresponding
kernel.That's all i always do."
Which kernel is he referring to is it Dom0?
because when I ran make xen it worked without any problem and I can see in my
/usr/src/xen-4.0.1/dist/install/boot
Quote:
xen-4.0.1.gz xen-4.0.gz xen-4.gz xen.gz xen-syms-4.0.1
but I am not clear with how to save this git pull each time.
I have given a link which I think is a solution to my problem but I am not able understand what they have mentioned.Can some one explain that.
 
Old 03-06-2011, 06:50 PM   #2
andrewthomas
Senior Member
 
Registered: May 2010
Location: Chicago Metro
Distribution: Arch, Gentoo, Slackware
Posts: 1,690

Rep: Reputation: 312Reputation: 312Reputation: 312Reputation: 312
I don't use a xen kernel, but I compile git kernels all the time. You probably should post the script that you are using.

What you want to do is not remove the sources every build.

Quote:
Originally Posted by tkmsr View Post
Code:
   + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
   + mkdir linux-2.6-pvops.git.tmp
   + rmdir linux-2.6-pvops.git.tmp
   + git clone -o xen -n
git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
linux-2.6-pvops.git.tmp
   Initialized empty Git repository in
/usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
   remote: Counting objects: 1941611, done.
   remote: Compressing objects: 100% (319127/319127), done.
remote: Total 1941611 (delta 1614302), reused 1930655 (delta 1604595)
**Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s, done.**
You could have a build directory and clean sources directory.

I don't know if they rebase the xen kernel often, if they don't you could just
to begin
Code:
mkdir build
cd build 
git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git clone xen xen-build
cd xen-build 
... do your magic here
then to update
Code:
cd build
rm -rf xen-build
git pull
git clone xen xen-build
cd xen-build
...do whatever you need to do
As I said, I am don't really know much about the xen kernel.

For instance, I don't know what branch you are building (master?)

If you would post the script, you certainly will get some help.

It defeats the purpose of using git to get the kernel source if you are removing and downloading the entire source each build.
 
Old 03-06-2011, 10:39 PM   #3
tkmsr
Member
 
Registered: Oct 2006
Distribution: Ubuntu,Open Suse,Debian,Mac OS X
Posts: 798

Original Poster
Rep: Reputation: 39
Quote:
Originally Posted by andrewthomas View Post
I don't use a xen kernel, but I compile git kernels all the time. You probably should post the script that you are using.
Well I am not using any script.
I have some doubt with respect to usage of git.
Quote:
Originally Posted by andrewthomas View Post
I don't know if they rebase the xen kernel often, if they don't you could just
to begin
mkdir build
cd build
git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git clone xen xen-build
cd xen-build
... do your magic here
then to update
With respect to above suggestion of your the Makefile that comes with Xen is here
http://lxr.xensource.com/lxr/source/Makefile
Quote:
Originally Posted by andrewthomas View Post
cd build
rm -rf xen-build
Want to know if xen-build is a command or you just gave an example to me.
Quote:
Originally Posted by andrewthomas View Post
git pull
git clone xen xen-build
cd xen-build
...do whatever you need to do[/CODE]
Quote:
Originally Posted by andrewthomas View Post
As I said, I am don't really know much about the xen kernel.

For instance, I don't know what branch you are building (master?)
I really do not understand git what I found is following happens each time I do a make world
Code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
and at each build it will download 900 Mb.Which is very costly for my internet.
Well it will help me if you can explain the above portion I have been trying to understand Xen and it has been very confusing.
What is the meaning of branches in git.
Quote:
git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x
The Xen page which talks such things is here
http://wiki.xensource.com/xenwiki/XenParavirtOps
Quote:
Originally Posted by andrewthomas View Post
If you would post the script, you certainly will get some help.
Ok here is Xen source code
http://lxr.xensource.com/lxr/source
and what I found is the lines responsible for the pull are here
http://lxr.xensource.com/lxr/source/....git-clone#L25
Quote:
Originally Posted by andrewthomas View Post
It defeats the purpose of using git to get the kernel source if you are removing and downloading the entire source each build.
You are right that is my problem I do not have such an internet connection to git clone 900 Mb each time when I build.
What I did is now by the time I saw your message I commented out following lines from
buidlconfig/src.git-clone directory of Xens tar ball.
Code:
  if ! [ -d $(LINUX_SRCDIR) ]; then \
 26                 rm -rf $(LINUX_SRCDIR) $(LINUX_SRCDIR).tmp; \
 27                 mkdir $(LINUX_SRCDIR).tmp; rmdir $(LINUX_SRCDIR).tmp; \
 28                 $(GIT) clone -o $(XEN_GIT_ORIGIN) -n $(XEN_LINUX_GIT_URL) $(LINUX_SRCDIR).tmp; \
 29                 (cd $(LINUX_SRCDIR).tmp; git checkout -b $(XEN_LINUX_GIT_LOCALBRANCH) $(XEN_LINUX_GITREV) ); \
 30                 mv $(LINUX_SRCDIR).tmp $(LINUX_SRCDIR); \
 31         fi
So now the git pull does not happen each time.But I do not know if above is the correct thing to do.

Last edited by tkmsr; 03-06-2011 at 10:43 PM.
 
Old 03-07-2011, 09:07 AM   #4
andrewthomas
Senior Member
 
Registered: May 2010
Location: Chicago Metro
Distribution: Arch, Gentoo, Slackware
Posts: 1,690

Rep: Reputation: 312Reputation: 312Reputation: 312Reputation: 312
The first question I have is why are you making world?

Quote:
@echo ' world - clean everything, delete guest kernel build'
this is why it cleans out the directory and you have to clone the entire git repo every time.

Downloading the git tree

Code:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
$ cd linux-2.6-xen
Changing the branch: You most probably want to use the "xen/stable-2.6.32.x" branch, so do this:
Code:
$ cd linux-2.6-xen
$ git reset --hard
$ git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x
Branch xen/stable-2.6.32.x set up to track remote branch refs/remotes/origin/xen/stable-2.6.32.x.
Switched to a new branch "xen/stable-2.6.32.x"
$ git pull
$ git log | less
Later when you want to update the tree use:
Code:
$ cd linux-2.6-xen
$ make clean
$ git pull
Example how to compile/build the pv_ops dom0 kernel:
Code:
cd linux-2.6-xen
make clean
cp -a .config .config-old
wget -O .config <http_url_to_config_file>
make oldconfig
make menuconfig (if you need to change something)
make bzImage
make modules
make modules_install

# in the following lines replace "version" with the actual kernel version you're compiling.
cp -a .config /boot/config-version
cp -a System.map /boot/System.map-version
cp -a arch/x86/boot/bzImage /boot/vmlinuz-version

# And then generate initrd/initramfs image for your dom0 kernel, example for Fedora/RHEL/CentOS:
mkinitrd -f /boot/initrd-version.img version
 
Old 03-07-2011, 11:09 AM   #5
tkmsr
Member
 
Registered: Oct 2006
Distribution: Ubuntu,Open Suse,Debian,Mac OS X
Posts: 798

Original Poster
Rep: Reputation: 39
Quote:
Originally Posted by andrewthomas View Post
The first question I have is why are you making world?
this is why it cleans out the directory and you have to clone the entire git repo every time.
Downloading the git tree
Ok thanks for this one I was not able to understand based on some blogs wiki etc I was doing it.
Quote:
Originally Posted by andrewthomas View Post
Changing the branch: You most probably want to use the "xen/stable-2.6.32.x" branch, so do this:
$ cd linux-2.6-xen
$ git reset --hard
$ git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x
Branch xen/stable-2.6.32.x set up to track remote branch refs/remotes/origin/xen/stable-2.6.32.x.
Switched to a new branch "xen/stable-2.6.32.x"
$ git pull
$ git log | less
Later when you want to update the tree use:
$ cd linux-2.6-xen
$ make clean
$ git pull
Ok I am not that used to with git so I will search that out.

I am clear with kernel compile process.I was not clear with make world part that you made clear.
Basically I think process of compiling a pv-ops dom0 kernel is same as compiling a normal kernel.
Xen I need to compile separately.
I was not clear with compiling of Xen that is where every thing was giving me confusion as make world etc was mentioned here which gave rise to all the confusion http://wiki.xensource.com/xenwiki/Xen4.0

Last edited by tkmsr; 03-07-2011 at 11:11 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to understand user time, sys time, wait time, idle time of CPU guixingyi Linux - Server 1 08-24-2010 10:10 AM
How can I reduce ntp server time? scbops Linux - General 4 07-02-2008 03:51 AM
how to reduce boot time raghuvamsi Linux - General 4 06-05-2008 01:40 PM
how to reduce time out time for "connect" kkpal Linux - Newbie 1 02-02-2008 02:02 PM
howto reduce boot time? provkitir Debian 5 11-04-2004 02:29 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud

All times are GMT -5. The time now is 05:10 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration