LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 04-18-2010, 05:14 PM   #1
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Rep: Reputation: 0
Problem with sock_create in KLM


Hello, i am dealing with studiing kernel module. As example i am using block device from lwn
http://lwn.net/Articles/58719/

and so far so good, but i am trying to create simple network disk (just for test, ...)

I found this article.

http://www.linuxjournal.com/node/7660

and this example is working, when aplied in init function, but when i use it instead of read / write, kernel freezes or in better case, sends some warning.(listed here)

Code:
Apr 18 22:30:54 Server1 kernel: [ 504.079823] rxdrv:<4>------------[ cut here ]------------
Apr 18 22:30:54 Server1 kernel: [ 504.143539] WARNING: at kernel/softirq.c:141 local_bh_enable+0x27/0x71()
Apr 18 22:30:54 Server1 kernel: [ 504.143539] Modules linked in: rxd(+) vboxvfs nls_base ipv6 loop parport_pc parport pcspkr serio_raw psmouse snd_intel8x0 i2c_piix4 i2c_core snd_ac97_codec ac97_bus battery ac snd_pcm snd_timer snd soundcore snd_page_alloc button vboxguest evdev ext3 jbd mbcache ide_cd_mod cdrom ide_disk ata_generic libata scsi_mod dock floppy ehci_hcd ohci_hcd piix ide_pci_generic pcnet32 mii usbcore ide_core thermal processor fan thermal_sys [last unloaded: scsi_wait_scan]
Apr 18 22:30:54 Server1 kernel: [ 504.143539] Pid: 3171, comm: insmod Tainted: G W 2.6.26-2-686 #1
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] warn_on_slowpath+0x40/0x66
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] __alloc_pages_internal+0xb5/0x34e
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] local_bh_enable+0x27/0x71
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] local_bh_enable+0x27/0x71
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] lock_sock_nested+0x84/0x8c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] inet_stream_connect+0x1a/0x1fd
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] rx_request+0xfe/0x178 [rxd]
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] __generic_unplug_device+0x1a/0x1c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] generic_unplug_device+0x1e/0x33
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] blk_unplug+0x55/0x5c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] sync_page+0x0/0x36
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] block_sync_page+0x2f/0x30
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] sync_page+0x2e/0x36
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] __wait_on_bit_lock+0x2a/0x52
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] __lock_page+0x4e/0x54
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] wake_bit_function+0x0/0x3c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] read_cache_page_async+0x9e/0xf8
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] blkdev_readpage+0x0/0xc
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] adfspart_check_ICS+0x0/0x14c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] read_cache_page+0xa/0x3f
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] read_dev_sector+0x26/0x60
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] adfspart_check_ICS+0x0/0x14c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] adfspart_check_ICS+0x20/0x14c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] printk+0x14/0x18
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] adfspart_check_ICS+0x0/0x14c
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] rescan_partitions+0x110/0x249
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] do_open+0x1e6/0x28d
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] __blkdev_get+0x63/0x6e
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] blkdev_get+0xa/0xc
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] register_disk+0xc9/0x115
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] add_disk+0x2c/0x6b
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] exact_match+0x0/0x7
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] exact_lock+0x0/0xd
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] init_rxd+0x126/0x151 [rxd]
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] blocking_notifier_call_chain+0x9/0xc
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] sys_init_module+0x187a/0x19ea
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] blk_init_queue+0x0/0x8
Apr 18 22:30:54 Server1 kernel: [ 504.143539] [] syscall_call+0x7/0xb
Apr 18 22:30:54 Server1 kernel: [ 504.143539] =======================
Apr 18 22:30:54 Server1 kernel: [ 504.143539] ---[ end trace 4eaa2a86a8e2da2 ]---
my code:

Code:
static void sbd_transfer(struct sbd_device *dev, unsigned long sector,
unsigned long nsect, char *buffer, int write)
{
long long offset = sector*hardsect_size;
unsigned long nbytes = nsect*hardsect_size;

if ((offset + nbytes) > dev->size) {
//printk (KERN_WARNING "sbd: Beyond-end write (%ld %ld)\n", offset, nbytes);
return;
}
if (write){

memcpy(dev->data + offset, buffer, nbytes);

}
else{

struct sockaddr_in servaddr;
int r = -1;
struct socket *control= NULL;
char *response = kmalloc(256,GFP_KERNEL);
unsigned short port;

r = sock_create(PF_INET, SOCK_STREAM,
IPPROTO_TCP, &control);

if(r!=0){
printk(KERN_WARNING "sbd: socketcreation \n" );
return;
}
port = htons(5902);
memset(&servaddr,0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = port;
servaddr.sin_addr.s_addr = htonl(0x10207964); // IP

r = control->ops->connect(control,
(struct sockaddr *) &servaddr,
sizeof(servaddr), O_RDWR);

if(r!=0){
printk(KERN_WARNING "sbd: socket not working, ending\n");
return;
}

sock_release(control);
kfree(response);

//memcpy(buffer, dev->data + offset, nbytes);

}
}
This code causes upper error even if there is no socket->opt->connect

This function is called several times while loading module (for reading)
but I do not know, why there is that bug.
Thank you for response in advance. i can post whole code via mail.

I am running debian lenny kernel 2.6.26-2-686

Pavel
 
Old 04-19-2010, 03:54 PM   #2
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,696

Rep: Reputation: 232Reputation: 232Reputation: 232
It seems that you have disabled interrupts one time too many. I can't see it in the code you provided. Re-check all the places where you enable and disable interrupts, especially check all error conditions. Also make sure you didn't make modifications to the kernel code outside of your module.
 
Old 04-20-2010, 02:35 AM   #3
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Original Poster
Rep: Reputation: 0
Thank you for reply, actually I don't see any interupt disabling in the code.
I am providing whole code bellow - it is only simple block device from lwn, and added network communication instead of memcpy that should read from memory.


Code:
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/init.h>

#include <linux/kernel.h> /* printk() */
#include <linux/fs.h>     /* everything... */
#include <linux/errno.h>  /* error codes */
#include <linux/types.h>  /* size_t */
#include <linux/vmalloc.h>
#include <linux/genhd.h>
#include <linux/blkdev.h>
#include <linux/hdreg.h>


//includes for bsd sockets;

#include <linux/net.h>
#include <net/sock.h>
#include <linux/tcp.h>
#include <linux/in.h>
#include <asm/uaccess.h>
#include <linux/file.h>
#include <linux/socket.h>
#include <linux/smp_lock.h>
#include <linux/slab.h>




MODULE_LICENSE("Dual BSD/GPL");

static int major_num = 0;
module_param(major_num, int, 0);
static int hardsect_size = 512;
module_param(hardsect_size, int, 0);
static int nsectors = 32768;  /* How big the drive is */ 
module_param(nsectors, int, 0);



/*
 * We can tweak our hardware sector size, but the kernel talks to us
 * in terms of small sectors, always.
 */
#define KERNEL_SECTOR_SIZE 512

/*
 * Our request queue.
 */
static struct request_queue *Queue;

/*
 * The internal representation of our device.
 */
static struct sbd_device {
    unsigned long size;
    spinlock_t lock;
    u8 *data;
    struct gendisk *gd;
} Device;





/*
 * Handle an I/O request.
 */
static void sbd_transfer(struct sbd_device *dev, unsigned long sector,
unsigned long nsect, char *buffer, int write)
{
long long offset = sector*hardsect_size;
unsigned long nbytes = nsect*hardsect_size;

if ((offset + nbytes) > dev->size) {
	printk (KERN_WARNING "sbd: Beyond-end write (%lld %ld)\n", offset, nbytes);
	return;
}

if (write){

	memcpy(dev->data + offset, buffer, nbytes);

}
else{

	struct sockaddr_in servaddr;
	int r = -1;
	struct socket *control= NULL;
	char *response = kmalloc(256,GFP_KERNEL);
	unsigned short port;

	r = sock_create(PF_INET, SOCK_STREAM,
		IPPROTO_TCP, &control);

	if(r!=0){
		printk(KERN_WARNING "sbd: socketcreation \n" );
		return;
	}
	port = htons(5902);
	memset(&servaddr,0, sizeof(servaddr));
	servaddr.sin_family = AF_INET;
	servaddr.sin_port = port;
	servaddr.sin_addr.s_addr = htonl(0x0A207964); // IP

	r = control->ops->connect(control,
	(struct sockaddr *) &servaddr,
	sizeof(servaddr), O_RDWR);

	if(r!=0){
		printk(KERN_WARNING "sbd: socket not working, ending\n");
	return;
	}

	sock_release(control);
	kfree(response);

//memcpy(buffer, dev->data + offset, nbytes);

	}
}


static void sbd_request(request_queue_t *q)
{
    struct request *req;

    while ((req = elv_next_request(q)) != NULL) {
	if (! blk_fs_request(req)) {
	    printk (KERN_NOTICE "Skip non-CMD request\n");
	    end_request(req, 0);
	    continue;
	}
	sbd_transfer(&Device, req->sector, req->current_nr_sectors,
			req->buffer, rq_data_dir(req));
	end_request(req, 1);
    }
}



/*
 * Ioctl.
 */
int sbd_ioctl (struct inode *inode, struct file *filp,
                 unsigned int cmd, unsigned long arg)
{
	long size;
	struct hd_geometry geo;

	switch(cmd) {
	/*
	 * The only command we need to interpret is HDIO_GETGEO, since
	 * we can't partition the drive otherwise.  We have no real
	 * geometry, of course, so make something up.
	 */
	    case HDIO_GETGEO:
		size = Device.size*(hardsect_size/KERNEL_SECTOR_SIZE);
		geo.cylinders = (size & ~0x3f) >> 6;
		geo.heads = 4;
		geo.sectors = 16;
		geo.start = 4;
		if (copy_to_user((void *) arg, &geo, sizeof(geo)))
			return -EFAULT;
		return 0;
    }

    return -ENOTTY; /* unknown command */
}




/*
 * The device operations structure.
 */
static struct block_device_operations sbd_ops = {
    .owner           = THIS_MODULE,
    .ioctl	     = sbd_ioctl
};

static int __init sbd_init(void)
{
/*
 * Set up our internal device.
 */

	Device.size = nsectors*hardsect_size;
    spin_lock_init(&Device.lock);
    Device.data = vmalloc(Device.size);
    if (Device.data == NULL)
	return -ENOMEM;
	

/*
 * Get a request queue.
 */
    Queue = blk_init_queue(sbd_request, &Device.lock);
    if (Queue == NULL)
	    goto out;
    blk_queue_hardsect_size(Queue, hardsect_size);
/*
 * Get registered.
 */
    major_num = register_blkdev(major_num, "sbd");
    if (major_num <= 0) {
	printk(KERN_WARNING "sbd: unable to get major number\n");
	goto out;
    }
/*
 * And the gendisk structure.
 */
    Device.gd = alloc_disk(16);
    if (! Device.gd)
	goto out_unregister;
    Device.gd->major = major_num;
    Device.gd->first_minor = 0;
    Device.gd->fops = &sbd_ops;
    Device.gd->private_data = &Device;
    strcpy (Device.gd->disk_name, "sbd0");
    set_capacity(Device.gd, nsectors*(hardsect_size/KERNEL_SECTOR_SIZE));
    Device.gd->queue = Queue;
    add_disk(Device.gd);
	printk(KERN_WARNING "sbd: Loaded, capacity %ld\n", Device.size);

    return 0;

  out_unregister:
    unregister_blkdev(major_num, "sbd");
  out:
    vfree(Device.data);
    return -ENOMEM;
}

static void __exit sbd_exit(void)
{
    del_gendisk(Device.gd);
    put_disk(Device.gd);
    unregister_blkdev(major_num, "sbd");
    blk_cleanup_queue(Queue);
    vfree(Device.data);
	printk(KERN_WARNING "sbd: Unloaded\n");
	
}
	
module_init(sbd_init);
module_exit(sbd_exit);


Thank you for help
Pavel

Last edited by payanek; 04-20-2010 at 02:39 AM. Reason: CODE is ignored - hopefully now correct
 
Old 04-21-2010, 03:13 PM   #4
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,850

Rep: Reputation: 161Reputation: 161
Maybe you create too many socket, one block request creates one socket.
 
Old 04-21-2010, 05:18 PM   #5
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Original Poster
Rep: Reputation: 0
I know, but I guess, that requests are served one by one from queue. Actually i tried to create one extra spinlock and lock execution of read section, but no change.

there was sequence :
lock
sock_create
sock_release
unlock
 
Old 04-25-2010, 02:48 PM   #6
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Original Poster
Rep: Reputation: 0
ok, once more I am here,
I guess (please, correct me), that problem is, that request processing is atomic operation, so I cannot switch content and that can cause problem with sock_create (even sock_create_kern)

If I am right, how to overcome this feature ?
Is there good idea to create one kernel thread and send there requests and call end_request function from that thread ?

thank you for hints.
Pavel

Last edited by payanek; 04-28-2010 at 02:43 AM. Reason: new idea :)
 
Old 04-28-2010, 03:34 PM   #7
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,850

Rep: Reputation: 161Reputation: 161
You can use kernel workqueue mechanism to wake up your waiting thread.
 
Old 04-28-2010, 03:49 PM   #8
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Original Poster
Rep: Reputation: 0
that seems to be interesting and usefull, thank you for hint, tommorow I will try it.
 
Old 04-30-2010, 02:21 AM   #9
payanek
LQ Newbie
 
Registered: Apr 2010
Location: Czech Republic
Distribution: Debian, Ubuntu
Posts: 6

Original Poster
Rep: Reputation: 0
Dear friends, thank everyone for help. Now, it seems to be working. Actually, I dropped Idea to comunicate with many oher clients directly from my module. Now I am communicating with userspace program, that is maitaining other communication.

Using working queue and worker thread to manage requests.
Great inspiration is nbd (network block device) whose source is directly in kernel.

Thank all of you for hints.

Pavel
 
  


Reply

Tags
kernel, linux, module, socket



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] KLM packet handler soft lockup on CPU: 0 yaplej Linux - Kernel 1 11-12-2009 06:03 PM
KLM Flight Entertainment System tokenhost Linux - General 3 03-05-2007 12:54 PM
Sound Card problem(every time i install linux i have diffirent hardware problem) jacka1l Linux - Newbie 7 08-11-2005 06:10 AM
perl problem? apache problem? cgi problem? WorldBuilder Linux - Software 1 09-17-2003 07:45 PM
How to PERMANENTLY Remove a KLM? RWild Linux - General 7 08-31-2003 11:54 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 04:29 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration