Swinging SlackHammer in a different direction...

hitest · 08-17-2009, 08:28 AM

Quote:

Originally Posted by GazL

I'm not even sure it has merit. If one was to run the DragonFly kernel and filesystem, then you wouldn't be able to use stuff that's tightly integrated with the linux kernel such as iptables, lvm, md-adm and all that stuff that makes linux linux. You could potentially port some of slackware's userspace packages over but they'd probably be available BSD native anyway. You could customise the Dragonfly startup scripts and suchlike to be more slackware like, but it would necessarily still be more Dragonfly than Slackware.

I really don't understand the point behind the suggestion. If you want to run Dragonfly and Hammer, then run them. I can see no value in attempting to make Dragonfly look like Slackware or use parts of slackware or it's application packages.

I was being kind, Gazl. I am a BSD user that is why I said it *probably* has merit rather than it *does* have merit. I just don't know. I'm with you; I am skeptical about this project. That is why we called for a working system. That is how it is done in the open source community, you build a new system, make an announcement, then invite replication and testing by your peers.
Agreed. I see no point to this system.
I've said my piece about this. I don't want to get in trouble.

onebuck · 08-17-2009, 08:44 AM

Hi,

Hitest, I really don't think the mods are going to get you for presenting a valid argument.

I personally don't think this can be adapted to the GNU/Linux kernel from the BSD kernel without major inclusion from the BSD kernel. It's not as if you can just cut & paste from one to the other.

There is so much that would have to be re-written to adapt this. Possibly creation of the filesystem using GNU/Linux would be far easier but still for a enterprise need. Why? The mega-T drives are not going to be used by a general user of Slackware over multiple systems via mirroring. Plus if a enterprise user is wanting the cluster wide support that Hammer is to provide to the systems will require decent network bandwidth for your systems;

Quote:

excerpt from 'HAMMER Filesystem Design';

HAMMER's approach to redundancy is logical replication of the entire filesystem. That is, wholely independant copies operating on different machines in different locations. Ultimately HAMMER's mirroring features will be used to further our clustering goals. The major goal of this project is transparent clustering and a major requirement for that is to have a multi-master replicated environment. That is the role HAMMER will eventually fill. We wont have multi-master in 2.0, but there's a good chance we will have it by the end of next year."

I agree the OP should work on getting something going instead of dream factoring something in hopes to seed someone else to go down the yellow brick road.

brianL · 08-17-2009, 08:55 AM

I'm no 1337 haxxor like Shingoshi, but this seems like another attempt by him to muck up and mongrelise Slackware.

GazL · 08-17-2009, 09:14 AM

Quote:

Originally Posted by hitest

I was being kind, Gazl.
...snip...
I've said my piece about this. I don't want to get in trouble.

Yep, I understood that. My reply wasn't really aimed at you hitest, that quote just seemed like a good jump-off point for what I wanted to say. On reflection, it would have probably been fairer not to drag you back into it by quoting you. Sorry.

Anyway, I've said my piece too so I won't post anything further on this either.

hitest · 08-17-2009, 09:26 AM

Quote:

Originally Posted by GazL

Yep, I understood that. My reply wasn't really aimed at you hitest, that quote just seemed like a good jump-off point for what I wanted to say. On reflection, it would have probably been fairer not to drag you back into it by quoting you. Sorry.

Anyway, I've said my piece too so I won't post anything further on this either.

No need to apologize, Gazl.

Your comments allowed me to finish stating the rest of my argument.
Thanks for your feedback, onebuck. I'm glad that you think I wasn't being too harsh.

easuter · 08-17-2009, 12:12 PM

I skimmed over the thread and I didn't see anyone mention this (so I apologize if it has already been said): Shingoshi, wouldn't it make more sense to implement the Hammer filesystem for the Linux kernel natively, instead of doing backwards somersaults trying to make a Dragonfly and Slackware mix?

Shingoshi · 08-17-2009, 12:29 PM

Quote:

Originally Posted by easuter

I skimmed over the thread and I didn't see anyone mention this (so I apologize if it has already been said): Shingoshi, wouldn't it make more sense to implement the Hammer filesystem for the Linux kernel natively, instead of doing backwards somersaults trying to make a Dragonfly and Slackware mix?

Your question was answered here: http://www.linuxquestions.org/questi...61#post3628061

There is NO working implementation of the Hammer filesystem for Linux. The current Hammer for Linux is READ ONLY. The person who wanted this initially, wanted WRITE capabilities as well. My suggestion was to simply short-circuit the previous plan by avoiding using the Linux kernel altogether. Essentially I've cut out the entire "switching back to Linux" from the previous plan. What you're left with is using the Dragonfly kernel as it should be. Because my objective in the future is to have a cluster-native kernel, which Dragonfly is developing.

I've waited on Linux for too long to implement clustering as a native feature. It could have been done years ago with openMosix. Instead, there are now numerous projects all headed in different directions. When Dragonfly succeeds in clustering their kernel, IT will become the standard against which all other systems are measured. I'm simply gambling on the likely and soon success of the Dragonfly project. My priority isn't to run a Linux kernel. My priority is to have clustering.

Running Slackware on top of the Dragonfly kernel would simply be another port of Slackware. Which I have dubbed as SlackHammer.

Shingoshi

easuter · 08-17-2009, 12:52 PM

Quote:

Originally Posted by Shingoshi

Your question was answered here: http://www.linuxquestions.org/questi...61#post3628061

There is NO working implementation of the Hammer filesystem for Linux. The current Hammer for Linux is READ ONLY. The person who wanted this initially, wanted WRITE capabilities as well. My suggestion was to simply short-circuit the previous plan by avoiding using the Linux kernel altogether. Essentially I've cut out the entire "switching back to Linux" from the previous plan. What you're left with is using the Dragonfly kernel as it should be. Because my objective in the future is to have a cluster-native kernel, which Dragonfly is developing.

I've waited on Linux for too long to implement clustering as a native feature. It could have been done years ago with openMosix. Instead, there are now numerous projects all headed in different directions. When Dragonfly succeeds in clustering their kernel, IT will become the standard against which all other systems are measured. I'm simply gambling on the likely and soon success of the Dragonfly project. My priority isn't to run a Linux kernel. My priority is to have clustering.

Running Slackware on top of the Dragonfly kernel would simply be another port of Slackware. Which I have dubbed as SlackHammer.

Shingoshi

Why don't you just run Dragonfly then, and skip the bastardization of Slackware altogether?
Seems like the simplest possible solution to whatever problem you are trying to solve...

cwwilson721 · 08-17-2009, 01:20 PM

Quote:

Originally Posted by Shingoshi

... I already clearly stated I DON'T have a system to do this on.....So I have nothing else to say about this to anyone who thinks they have the right to be critical or disapprove of the idea.

Shingoshi

[Edit: I'm not bashing, just ticked about the attitude of the OP. The OP may be right, or may be wrong. BUT THE ATTITUDE...]

Erm...correct me if I'm wrong....

You claim this works.
Nobody has the right to critical about this.

Ok... But

YOU HAVEN'T DONE THIS.
READ THE ABOVE

I think that's a huge hole in whatever passes for your logic.

All you're doing is repackaging a General Motors idea: SELL THE CAR. So what if we don't build it? All we want is the idea of the car...And money. While you aren't in it for the financial reasons, you DON'T HAVE A CAR TO SELL.

Using your logic, there is a magic OS that is developed, that pixies deliver, and NEVER CRASHES (And it's NOT M$Vista...). There. Now it exists. And I know it does..But I never installed or used it, but I know it works. Why? Because I said so.

You also cried in another, NAMELESS POST, about everybody bashing you for your views. I guess the entire community is wrong, and you're right, again.

Too bad. You're being criticized.

If you EVER wish to be taken seriously, ACCOMPLISH SOMETHING.
Take your own advice, and DO IT.

THEN tell us about it. And ask for our opinion. And, most of all, take our advise/views in a CONSTRUCTIVE way.

Until then, you will probably ONLY get critical comments.

BTW, anyone want the new OS?

onebuck · 08-17-2009, 02:57 PM

Hi,

Quote:

Originally Posted by Shingoshi

Your question was answered here: http://www.linuxquestions.org/questi...61#post3628061

No, it wasn't answered by your post in another thread. Just more bloviating!

Quote:

Originally Posted by Shingoshi

There is NO working implementation of the Hammer filesystem for Linux. The current Hammer for Linux is READ ONLY. The person who wanted this initially, wanted WRITE capabilities as well. My suggestion was to simply short-circuit the previous plan by avoiding using the Linux kernel altogether. Essentially I've cut out the entire "switching back to Linux" from the previous plan. What you're left with is using the Dragonfly kernel as it should be. Because my objective in the future is to have a cluster-native kernel, which Dragonfly is developing.

I've waited on Linux for too long to implement clustering as a native feature. It could have been done years ago with openMosix. Instead, there are now numerous projects all headed in different directions. When Dragonfly succeeds in clustering their kernel, IT will become the standard against which all other systems are measured. I'm simply gambling on the likely and soon success of the Dragonfly project. My priority isn't to run a Linux kernel. My priority is to have clustering.

Running Slackware on top of the Dragonfly kernel would simply be another port of Slackware. Which I have dubbed as SlackHammer.

Shingoshi

I think you will be waiting a longer time because of the way 'Hammer' relies on Dragonfly;

Quote:

excerpt from 'THE HAMMER FILESYSTEM
Matthew Dillon, 21 June 2008' white paper;

Hammer is part of a larger DragonFly clustering project and the intention is to
evolve the file-system to a point where multi-master replication can be done.
This feature clearly will not be available in the first release and also requires a
great deal of OS-level support for cache coherency and other issues. Hammer's
currently mirroring code only supports a single master, but can have any number
of slaves (mirroring targets). Ultimate up to 16 masters will be supported

If you think a R/W 'Hammer' will work for the 'GNU/Linux' kernel then read on;

Quote:

excerpt from 'THE HAMMER FILESYSTEM
Matthew Dillon, 21 June 2008' white paper;

If you've slogged through this whole document then you know that while the first
release of Hammer is very complete, there are also a ton of very cool features
still in the pipeline. I intend to support porting efforts and I'm gonna say that
doing so will probably require a lot of work on the source code to separate out
OS-specific elements. The only way to really make Hammer a multi-port filesystem
is to focus on splitting just four files into OS-specific versions. The whole
of “hammer_vnops.c”, the whole of “hammer_io.c”, a port-specific header file,
and an OS-specific source file supporting a Hammer-API for mount, unmount,
and management of the support threads (things like tsleep, wakeup, and
timestamps).
If we do not do this then porters are likely going to be left behind as work
continues to progress on Hammer. I am looking into placing the code in its own
repository (something other then CVS) so it can be worked on by many people.
Here the big porting gotchas people need to be on the lookout for:
• Inode numbers are 64 bits. I'm sorry, you can't cut it down to 32. Hammer
needs 64.
• Hammer uses 64 bit byte offsets for its buffer-cache interactions, but
accesses will always be in multiples of 16K (and aligned as such).
• Hammer uses 16K buffers and 64K buffers within the same inode, and will
mix 16K and 64K buffers on the same block device.
• Hammer's direct-io bypass creates a lot of potential conflicts between
buffer cache buffers of differing sizes and potentially overlapping the same
physical storage (between a buffer cache buffer associated with a front-end
vnode and one associated with a back-end block device). Lots of hacks
exist to deal with these.
• Hammer's DragonFly implementation uses advanced bioops which allows
Hammer to passively return the buffer to the kernel, but still have veto
power if the kernel wants to reuse or flush the buffer.
• Hammer uses DragonFly's advanced VNOPS interface which uses a
simplified name-cache-oriented API for name-space operations.
• Hammer expects the kernel to prevent name-space and data-space
collisions and to handle atomicy guarantees. For example if you try to
create a file called “B” and also rename “A” to “B” at the same time, the
kernel is expected to deal with the collision and not dispatch both
operations to the Hammer file-system at the same time.
Kernels which do not deal with atomicy and name-space guarantees will
need to implement a layer TO guarantee them before dropping into
Hammer's VNOPS code.

Shingoshi, the doors open for you to implement as you please. There's the challenge back at you to produce a working version for 'GNU/Linux'. Read the above referenced paper for 'Hammer', I have read these along with other information. You alone will not implement this for GNU/Linux. Why not just use 'UFS';

Quote:

excerpt from KernelTrap;

Quote: Benchmark Of The Filesystem
August 14, 2008 - 9:47pm

"Any benchmark is going to be a benchmark of the OS as much as it is going to be a benchmark of the filesystem. It's pretty hard to separate the two. ZFS is best tested on Open Solaris. UFS is best tested on FreeBSD, EXT3 is best tested on Linux, and HAMMER of course is best tested on DragonFly."

I don't expect you to get my reasoning for referencing the above quote. Most filesystems do perform better on their native OS not the same for implementations/ports.

Read the paper and come back with the operating filesystem for GNU/Linux.

Petri Kaukasoina · 08-17-2009, 03:02 PM

So, the idea is to port a Linux distribution to run on a kernel forked from FreeBSD. This is something similar: http://lwn.net/Articles/329556/ "Debian GNU/kFreeBSD". They are making a Debian distribution run on a FreeBSD kernel. GNU libc and lots of packages have been ported and the kernel is a FreeBSD kernel with a few patches. It's a good story to see how much work there is in this kind of project. Of course, it's now easier to do again.

onebuck · 08-17-2009, 04:26 PM

Hi,

Good article!

Shingoshi, a look at the potential risk and time to be spent in doing what was suggested here;

Quote:

excerpt from 'Debian GNU/kFreeBSD: one more step towards a universal operating system';

The project was first announced in the Debian Weekly News for February 22nd, 1999:
Someone proposed a Debian distribution based on FreeBSD. There was considerable debate on this topic. Most of the favorable opinions expressed were based on the argument that there should be a Debian distribution for as many open source UNIX variants as possible. This was countered with the argument that this would drastically increase the workload of the package maintainers.

As lightly stated a 'few packages';

Quote:

excerpt from 'Debian GNU/kFreeBSD: one more step towards a universal operating system';

Switching to the GNU libc port has brought better compatibility with the Debian packages and, once that happened, a lot of packages were able to be built without any changes. The project got the name Debian GNU/kFreeBSD. To summarize, Debian GNU/kFreeBSD is a port that consists of a GNU user space using the GNU C library and Debian package management and system tools on top of FreeBSD's kernel. The latest Debian GNU/kFreeBSD is based on the upstream FreeBSD 7.1 kernel with a few patches.

There's that damn kernel issue(s) again;

Quote:

excerpt from 'Debian GNU/kFreeBSD: one more step towards a universal operating system';
Wireless networking works in Debian GNU/kFreeBSD, but there are no tools to scan for networks yet. The problem is that NetworkManager is really tightly coupled to HAL, which was not available on FreeBSD. Jarno explains: "Though it has been originally designed for being portable across operating systems, a lot of kernel-specific code had to be written." However, now that HAL has been ported to FreeBSD, it should not be a big deal to get NetworkManager working. As upstream FreeBSD is doing the porting work (as a Google Summer of Code project), Debian GNU/kFreeBSD will surely get NetworkManager support in the near future.

I've spent all the time I'm going to spend on this issue.

Get this one point, not going to be done in the manner that you have been presenting.

Good Luck!

Shingoshi · 08-26-2009, 03:34 AM

I had stripped down my main server recently. And since I'm going to need a cluster system as soon as it becomes available, I now have to consider changing that other system over.

My primary considerations going into this is that I have had the files on that old system's disks since 2004. That's a long continuity to break up for a new install with something that'll have to remove everything beforehand. The current disks from that old system are formatted with XFS on RAID1. I had just recently purchased two Western Digital 1TB drives and transferred my system root over to them. But presently, it's the only alternative I have now. So I guess I'll have a lot of housekeeping to do, transferring files from that machine to this one. Once that's done, I can proceed with the installation of DragonflyBSD.

Shingoshi

akus · 08-26-2009, 01:37 PM

Quote:

Originally Posted by Shingoshi

I had stripped down my main server recently.

My primary considerations going into this is ..

Shingoshi

Thank you for an update. We'll keep our fingers crossed. If you need anything, do not hesitate to ask.

Shingoshi · 08-26-2009, 02:07 PM

Quote:

Originally Posted by akus

Thank you for an update. We'll keep our fingers crossed. If you need anything, do not hesitate to ask.

Because you would be one of the few who has actually responded in a positive manner. But the fact is, I still have to prepare myself for removing a set of system files I've had functioning since 2004. I guess I just have to get over it and move on.

Shingoshi