Python. NULL a pointer returned from a c funtion.

I figured out to use the gc library. But I'm still getting a single leak which is only 120bytes. I'm thinking it's the pointer not being NULLed.

How can I do that? Once I've run del var it's got nothing to point to.

That or am I going about this wrong. stuck.

#!/usr/bin/python3
# python bindings for libjmgeneral.so

import ctypes
from ctypes import *
import pathlib
import gc

lib = pathlib.Path().absolute() / "/usr/local/lib/libjmgeneral.so"
libjmgen_lib = ctypes.CDLL(lib)

def random_gen(a, b):
	return libjmgen_lib.random_gen(a, b)
  
def clear_ptr(a):
  del a
  
  
  
def prompt_input_getline(a):
  prompt_input_getline = libjmgen_lib.prompt_input_getline
  prompt_input_getline.argtypes = [c_char_p]
  prompt_input_getline.restype = c_char_p
  result = prompt_input_getline(a.encode('utf-8'))
  return result.decode('utf-8')
  
y = prompt_input_getline("lol")

print(y)

clear_ptr(y)

I guess libjmgeneral is your own lib. I have no any idea what is the problem. In the title you told null pointer is returned, in post you are speaking about a leak.
How did you detect that? What's happening at all? How is it related to gc? How does this lib work?

(I haven't actually ever combined C with Python code, so the below is just a guess from a quick look at the docs and some StackOverflow answers, but...)

If you want to deallocate your C allocated memory the Python garbage collector isn't going to do it for you. You have call the C function free() on it. I'm also pretty sure that the result.decode('utf-8') expression is making a copy of the string, so your y variable isn't holding a pointer to the original C allocated memory anyway. There is no point in calling clear_ptr or anything else of the sort on it.

If you want the Python garbage collector to handle the memory for you, then you have to allocate it with ctypes.create_string_buffer() or similar.

Originally Posted by pan64 In the title you told null pointer is returned

He's using NULL as a verb (not entirely standard, but it's equivalent to "clear" or "zero out").

I apologize. Posted tired and frustrated.

Let me break it down more. I used NULLing a pointer as a thing to do.

When I run the above Python code which pulls that function from my C library I've been working on I'm trying (expecting?) to return a pointer to a malloced string to the Python program. Ultimately with valgrind I'm getting a leak of 120bytes that tracks to that malloc. I'm assuming that the malloc is freed but the pointer is not set to NULL as I've seen that exact value before when I've forgotten to NULL something on C.

I'll look into the string_buffer thing. Thank you for that suggestion. Will post back when I get a chance.

You're still uppercasing "null" when you shouldn't be.

If you're writing C code, one must use "NULL" because that's what the language specification says.

If you're writing English, referencing the [lack of] value as "NULL" is acceptable, (so would be writing "null", except for potentially being ambiguous).

If you're writing English and using as a verb it should always be lowercase - i.e. "null", "nulled" "nulling", etc - writing "NULLing" is wrong, writing "NULL" for the verb is wrong and confuses people.


As for the issue itself, instead of describing what's happening, it may help to show the exact commands and output (and any relevant environment variables).

Current code as it sits

#!/usr/bin/python3
# python bindings for libjmgeneral.so

import ctypes
from ctypes import *
import pathlib
import gc

lib = pathlib.Path().absolute() / "/usr/local/lib/libjmgeneral.so"
libjmgen_lib = ctypes.CDLL(lib)

def random_gen(a, b):
	return libjmgen_lib.random_gen(a, b)
  
def prompt_input_getline(a):
  prompt_input_getline = libjmgen_lib.prompt_input_getline
  prompt_input_getline.argtypes = [c_char_p]
  prompt_input_getline.restype = c_char_p
  result = prompt_input_getline(a.encode('utf-8'))
  return result.decode('utf-8')
  

y = prompt_input_getline("lol")

print(y)

C functions called for prompt_input_getline

char *
getline_stdin_mem_alloc(void)
{
  // declare & initialize
  size_t buflen = 0;
  char * buffer = NULL;
  if (getline(&buffer,
              &buflen,
              stdin) == -1) {
    free(buffer); buffer = NULL;
  }
  
  if (buffer) {
    no_more_newline(buffer);
  }

  return(buffer);
}

char *
prompt_input_getline(const char * prompt)
{
  assert(prompt);
  
  // declare & initialize
  char * retval = NULL;

  // print & get input
  prompt_print(prompt);
  retval = getline_stdin_mem_alloc();
  if (retval) {
    no_more_newline(retval);
  }

  return(retval);
}

void
prompt_print(const char * prompt)
{
  assert(prompt);
  printf("%s --> ",
         prompt);
}

Full valgrind output here. https://paste.ubuntu.com/p/XwKnz7BJVk/

If I try to add any free call either directly to libc or via a function (which I'm not even sure works the way I think it does...

void clear_ptr(char * ptr)
{
  if (ptr) {
    free(ptr);
    ptr = NULL;
  }
}

I get a double free error. I've tinkered with

del a

before returning the result. It aborts on an empty pointer. If I run the

del a

after the output it leaves behind this memory (tail end of valgrind output)

definitely lost: 120 bytes in 1 blocks

If I call the clear_ptr function above I get a double free or corruption error.

My guess would be:

void free_ptr(char * ptr)
{
    free(ptr); // NOTE: free() already checks for NULL
}

def prompt_input_getline(a):
  prompt_input_getline = libjmgen_lib.prompt_input_getline
  prompt_input_getline.argtypes = [c_char_p]
  prompt_input_getline.restype = c_char_p
  result = prompt_input_getline(a.encode('utf-8'))
  strResult = result.decode('utf-8')
  free_ptr(result)
  return strResult

(I'm quite certain that setting the local variable ptr to NULL doesn't do anything useful, so I dropped that part and renamed the function from clear_ptr now that there is no clearing happening. Obviously it would make sense at this point to just call free() directly, but I'm not sure about the syntax for that, so I left it as a wrapper.)

sorry, probably I'm wrong, but where did you malloc anything for buffer?
I think you need to have a local buffer for getline_stdin_mem_alloc, and malloc a new buffer for the result and return the pointer to that.

Originally Posted by pan64 sorry, probably I'm wrong, but where did you malloc anything for buffer?

https://www.gnu.org/software/libc/ma...ine-Input.html

Function: ssize_t getline (char **lineptr, size_t *n, FILE *stream) [...] If you set *lineptr to a null pointer, and *n to zero, before the call, then getline allocates the initial buffer for you by calling malloc

Apologies for my NULL / null grammar errors earlier. Sorry to take so long to get back. Long day. This is where I'm at now. I've eliminated the definitely lost memory leak. I'm fairly certain what I'm getting now is out of my hands.

Current Python binding file.

#!/usr/bin/python3
# python bindings for libjmgeneral.so

import ctypes as ct
from ctypes import *
import pathlib

lib = pathlib.Path().absolute() / "/usr/local/lib/libjmgeneral.so"
local_lib = ct.CDLL(lib)

def random_gen(a, b):
	return local_lib.random_gen(a, b)

def clear_ptr(a):
  clear_ptr = local_lib.clear_ptr
  clear_ptr.argtypes = ct.c_void_p,
  clear_ptr.restype = None
  clear_ptr(a)

def prompt_input_getline(a):
  # prompt return
  promptget = local_lib.prompt_input_getline
  promptget.argtypes = [c_char_p]
  promptget.restype = ct.POINTER(ct.c_char_p)
  # get return
  promptret = promptget(a.encode('utf-8'))
  # cast void to char ptr
  retval = ct.cast(promptret, ct.c_char_p).value
  # free pointer memory
  clear_ptr(promptret)
  # return
  return(retval.decode('utf-8'))

y = prompt_input_getline("lol")

print(y)

C functions are the same as above.

Valgrind output.

valgrind -s --leak-check=full ./libjmgeneralpybind.py --show-leak-kinds=all
==16022== Memcheck, a memory error detector
==16022== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16022== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==16022== Command: ./libjmgeneralpybind.py --show-leak-kinds=all
==16022== 
lol --> rofl
b'rofl'
==16022== 
==16022== HEAP SUMMARY:
==16022==     in use at exit: 577,086 bytes in 169 blocks
==16022==   total heap usage: 2,270 allocs, 2,101 frees, 3,987,575 bytes allocated
==16022== 
==16022== 568 bytes in 1 blocks are possibly lost in loss record 10 of 126
==16022==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==16022==    by 0x224587: _PyObject_GC_NewVar (in /usr/bin/python3.10)
==16022==    by 0x24A2E6: ??? (in /usr/bin/python3.10)
==16022==    by 0x2629CD: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==16022==    by 0x24B45B: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==16022==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==16022==    by 0x24B45B: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==16022==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==16022==    by 0x24B26C: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==16022==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==16022==    by 0x24B26C: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==16022==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==16022== 
==16022== LEAK SUMMARY:
==16022==    definitely lost: 0 bytes in 0 blocks
==16022==    indirectly lost: 0 bytes in 0 blocks
==16022==      possibly lost: 568 bytes in 1 blocks
==16022==    still reachable: 576,518 bytes in 168 blocks
==16022==         suppressed: 0 bytes in 0 blocks
==16022== Reachable blocks (those to which a pointer was found) are not shown.
==16022== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==16022== 
==16022== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Is progress I suppose. Thank you for the direction and suggestions to all.

Originally Posted by jmgibson1981 promptget.argtypes = [c_char_p] promptget.restype = ct.POINTER(ct.c_char_p) # get return promptret = promptget(a.encode('utf-8')) # cast void to char ptr retval = ct.cast(promptret, ct.c_char_p).value

If I'm reading this right, you are first incorrectly declaring the return type of promptget as pointer-to-char-pointer (i.e., char**), and then casting the return value to the correct type (char*)?

(Does .value make a copy of the pointed-to value? I guess it must otherwise you'd have use-after-free which Valgrind should catch.)

You were right about the POINTER declarations. They do work fine with c_char. However that memory error coming from a malloc in python is still there. But it's not even a definitely lost.

valgrind -s --leak-check=full ./libjmgeneralpybind.py 
==33016== Memcheck, a memory error detector
==33016== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==33016== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==33016== Command: ./libjmgeneralpybind.py
==33016== 
/home/jason
==33016== 
==33016== HEAP SUMMARY:
==33016==     in use at exit: 576,142 bytes in 168 blocks
==33016==   total heap usage: 2,303 allocs, 2,135 frees, 4,151,253 bytes allocated
==33016== 
==33016== 568 bytes in 1 blocks are possibly lost in loss record 10 of 125
==33016==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==33016==    by 0x224587: _PyObject_GC_NewVar (in /usr/bin/python3.10)
==33016==    by 0x24A2E6: ??? (in /usr/bin/python3.10)
==33016==    by 0x2629CD: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==33016==    by 0x24B45B: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==33016==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==33016==    by 0x24B45B: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==33016==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==33016==    by 0x24B26C: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==33016==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==33016==    by 0x24B26C: _PyEval_EvalFrameDefault (in /usr/bin/python3.10)
==33016==    by 0x2629FB: _PyFunction_Vectorcall (in /usr/bin/python3.10)
==33016== 
==33016== LEAK SUMMARY:
==33016==    definitely lost: 0 bytes in 0 blocks
==33016==    indirectly lost: 0 bytes in 0 blocks
==33016==      possibly lost: 568 bytes in 1 blocks
==33016==    still reachable: 575,574 bytes in 167 blocks
==33016==         suppressed: 0 bytes in 0 blocks
==33016== Reachable blocks (those to which a pointer was found) are not shown

But still that is an improvement on my code. It's better and I learned something. I appreciate that very much. I'll keep tinkering to chase down this malloc issue above. But for now making progress on my python bindings nicely. Thank you all again.

*EDIT* Further testing confirms this result is in Python itself somehow. When I execute the binding file without any function calls, it just exits doing nothing with the exact same valgrind output as earlier in this post.