LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 01-22-2006, 08:25 AM   #1
slzckboy
Member
 
Registered: May 2005
Location: uk - Reading
Distribution: slack 10.2 kde 3.4.2 kernel 2.6.15
Posts: 452

Rep: Reputation: 30
why are strsep and strtok() to be avoided?


even their own man pages say not to use them?

Anyone know of a similar replacement for these functions,

some library of some sort ??

Pls don't make me write my own function lol

 
Old 01-22-2006, 10:37 AM   #2
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Well, you did read the man pages, didn't you? As it states, the problem is that those two functions work by assuming (as is, of course, correct for C) that the first input argument is a pointer to the address of a character array, and they work by modifying the contents of the character array and the address of the start of the array.

Specifically, consider this code:
Code:
#include <stdio.h>
#include <string.h>
int main()
{
  char *x, *y;
  int i = 0;
  x =(char *) malloc((size_t)128);
  strcpy(x,"This is a test;Testing 1, 2, 3;Done");
  printf ("x=%x\ni=%d: %s\n", &x, i, x);
  while (y = strsep(&x, ";")) {
    printf("y=\"%s\"\n", y);
    printf("x=%x\ni=%d: %s\n", &x, ++i, x);
  }
  return 0;
}
which produces this:
Code:
x=bfa6a244
i=0: This is a test;Testing 1, 2, 3;Done
y="This is a test"
x=bfa6a244
i=1: Testing 1, 2, 3;Done
y="Testing 1, 2, 3"
x=bfa6a244
i=2: Done
y="Done"
x=bfa6a244
i=3: (null)
Notice that &x does not change, whilst x[] does change.

The "problem" is that your input string will be modified as a "side effect" of using the function. In general, this kind of action produces code that is very hard to either test or debug. Among other thing, unless you've used a fixed size malloc call (as I did in the example) you may end up with memory leak problems.

In essence, those two functions "convert" the input string from a char[] array to a char[][] array, and return the next value of the "iterator" into the array of strings.

If you have a choice, consider using a C++ string object with well defined behavior.
 
Old 01-22-2006, 02:57 PM   #3
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
strtok still useful...

As PTrenholme pointed out, the problem with these two functions is that they're "non-reentrant": they assume that state will be preserved between invocations - an assumption that's decidedly NOT true in a multithreaded application.

Nevertheless, "strtok()" can be an extremely handy function. I would be *cautious* about using it - but I wouldn't necessarily *avoid* using. Especially if it happens to be to be the best solution for your particular code.

When any of my methods or functions use "strtok()", I always make sure:
a) consecutive calls to the string in question are always single-threaded
b) the code is clearly commented "non-reentrant: uses strtok()!"

IMHO .. PSM
 
Old 01-25-2006, 10:01 AM   #4
slzckboy
Member
 
Registered: May 2005
Location: uk - Reading
Distribution: slack 10.2 kde 3.4.2 kernel 2.6.15
Posts: 452

Original Poster
Rep: Reputation: 30
That makes sense.Thnks gents.

I did read the MAN page,but I think I just needed it explained in another way.

I think for my purpose I will just malloc a copy of the buffer to be parsed and then free the resource after I have extracted the information I need as intimated above.

Would someone mind explaining why there is a potential for memory leaks though ?
 
Old 01-25-2006, 03:41 PM   #5
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Quote:
Originally Posted by slzckboy
[snip]
Would someone mind explaining why there is a potential for memory leaks though ?
That was an offhand comment prompted from my experience with older C (and, in fact, B) programmers, who often neglected the call to free(). In fact, in the example code, I neglected to do so, myself.
 
Old 01-25-2006, 04:15 PM   #6
slzckboy
Member
 
Registered: May 2005
Location: uk - Reading
Distribution: slack 10.2 kde 3.4.2 kernel 2.6.15
Posts: 452

Original Poster
Rep: Reputation: 30
oh.. no worries.

well thnks again for your explanation.
 
Old 04-27-2009, 11:33 AM   #7
MrMichaelWill
LQ Newbie
 
Registered: Nov 2003
Posts: 10

Rep: Reputation: 0
Quote:
Originally Posted by slzckboy View Post
even their own man pages say not to use them?

Anyone know of a similar replacement for these functions,

some library of some sort ??

Pls don't make me write my own function lol

Unfortunately the manpage was updated by an amateur. The original author of the functions would certainly disagree :-)

The important information in summary (which you already partially got) is:
1. the strtok will modify the original data, which means that you have to make a copy of it before you work on it unless you don't care
2. the strtok function is not thread-safe, which means it relies on some global (within your program) status data, which works fine for your application unless you use multiple threads in the same process (pthreads). Then you want to use strtok_r instead, which should work fine.

The person leaving the discouraging and confusing comment in the manpage failed to point out an alternative.

I am using strtok_r successfully.

Michael
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
C using strtok to parse by quotes? zero79 Programming 3 03-07-2005 03:44 PM
strtok pantera Programming 2 12-19-2004 01:04 PM
free memory from strtok? swinchen Programming 1 09-08-2004 11:01 PM
strsep vs. strtok irfanhab Programming 1 05-01-2004 01:08 AM
Alternatives to strtok? jpbarto Programming 4 03-26-2004 01:20 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration