LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   partial serialize/deserialize a structure (https://www.linuxquestions.org/questions/programming-9/partial-serialize-deserialize-a-structure-486963/)

George2 09-29-2006 08:02 AM

Hi sundialsvcs,


Quote:

Originally Posted by sundialsvcs
I agree that it is generally wise to let the virtual-memory subsystem be your "very large disk file." Just be sensible about it. Use data-structures that allow the system to zoom in on a particular key with a minimum of memory accesses. The OS will automatically handle memory management for you and will do it very well.

My first purpose is to limit the amonut of memory my application will use, so that the application will not impact other applications. I think if we use OS built-in functions, which are transparent to users, to swap materials from/to memory/disk, my application may consume too much memory ...


regards,
George

George2 09-29-2006 08:07 AM

Hi exvor,


Quote:

Originally Posted by exvor
Not tying to hijack the thread but will i have issues with what your saying here.

2) making sure it can deal with a memory state that varies from when it was stored.


if its a structure it should always have the same size when writeing it from disk ?

I also have a question here ... a memory state means what? And could you provide a simple scenario to illustrate what means "that varies from when it was stored"?


regards,
George

George2 09-29-2006 08:09 AM

Thank you ta0kira!


Quote:

Originally Posted by ta0kira
If one structure holds a pointer to something else and the structure is "saved" then removed from memory (it would have to be for this whole thing to be useful), that which it pointed to might not be there when it gets loaded back into memory. This could happen if the pointer is to another structure (or a component of it) that was subsequently saved + removed. That's why I asked about pointers to other structures; if they're used, then each structure "might" require a "serial number" independent of the pointer which will have to be saved along with it, and those pointers to others will have to be converted to serial numbers, also (all of which would need to be managed by a cache of some sort.)
ta0kira

I understand your points now. My data structure does not contain any pointer to other data structures.


regards,
George

George2 09-29-2006 08:12 AM

Hi exvor,


Quote:

Originally Posted by exvor
Ohh my program doesent have pointers to other structures in the structures. but for the original question i can see how that would become a problem as a linked list has pointers to the other elements. When i was condisering doing something like what you were going to do i was going to save all the structures to the disk file and then rebuild the linked list from the data. I never got that out of the concept stage tho so there was probably alot of issues i never thought to work out.

There is no problem that you can *rebuild* the linked list. But how do you *rebuild* matters. If you simply save/load pointers to next/previous linked list node to/from disk. It will not work, since the nodes are not ensured to be in the same memory slot after reading again from disk.

So, you have to *rebuild* both the values and the link pointers.


regards,
George

George2 09-29-2006 08:13 AM

Thank you ta0kira!


Quote:

Originally Posted by ta0kira
I agree. Even if the end user has no VM, the program could take whatever space it was going to use as a cache and loop it as swap space.
ta0kira

I agree with you ta0kira that the VM solution works. But my first purpose is to limit the amonut of memory my application will use, so that the application will not impact other applications. I think if we use OS built-in functions, which are transparent to users, to swap materials from/to memory/disk, my application may consume too much memory ...


regards,
George

George2 09-29-2006 08:25 AM

Thank you for your quick sample, exvor!


Quote:

Originally Posted by exvor
I have acutally gotten my code to work properly now but im not sure its really what your trying to do. In my example it only works with a struct called datanode that gets passed to my read and write functions. Im not working with a list of stuff.

anyway here is my write function (it will be totaly dismantaled and rewritten so that It has more functionality.) in the new version it wont look for the correct place anymore but rather take a file postion from my list function and just write data using that file postion. hope this gets your ideas going tho.

Code:



#include<stdio.h>
#include<stdlib.h>
#include"cddat.h"


int addcd(DATANODE *infoadd,const char *database,int option)
{
    FILE *cddat;
    int number;
    int loop = 0;
    DATANODE *chek; 

    if ((cddat = fopen(database,"r+")) == NULL)
      {
      printf("\nError writeing to file!. func addcd\n");
      return 1;
      }

    number = infoadd->cdnum;
   
    if (option == 1)
    {
       


        fwrite(infoadd,sizeof(DATANODE),1,cddat);
    }
    else if (option == 2)
      {
       
        while(feof(cddat) == 0)
          {
            chek = malloc(DATANODE);
            fseek(cddat,loop * sizeof(DATANODE),SEEK_SET);
            fread(chek,sizeof(DATANODE),1,cddat);
            if (chek->cdnum == number)
                    {
                      free(chek);
                      fwrite(infoadd,sizeof(DATANODE),1,cddat);
                      return 0;
                    }
            else
                {
                  loop++;
                  free(chek); 
                }
            }
      return 2; /* This only occurs if the element is not in the file */

      }
    else
      { fclose(cddat);
        return 1;
      }
 
  if (fclose(cddat))
            {
        printf("\nError closeing file. func addcd\n");
        return 1;
      }
 
  else
    return 0;
}

as you can see checking for memory issues is not built in yet.


here is the lister by the way it acutally returns file pos now
Code:


#include<stdio.h>
#include<stdlib.h>
#include"cddat.h"


long int listfile(const char *basename)
{
  FILE *basefile;
  DATANODE *temphold;
  int discnum;
  int check; 
 
  int looper = 0;
 
  if ((basefile = fopen(basename,"r")) != NULL)
 {
 

  temphold = malloc(sizeof(DATANODE));
  printf("\nWhat disc would you like to access?\n");
  printf("DISK#=");
  scanf("%d",&discnum);
     
   
  do {
    fseek(basefile,looper * sizeof(DATANODE),SEEK_SET); 
  fread(temphold,sizeof(DATANODE),1,basefile);
  check = temphold->cdnum;
  printf("\ndiscnum %d, check %d\n",discnum,check);
  if (discnum == check)
      {

 
     
     
          printf("\nRecord # %d\n",temphold->cdnum);
          printf("+----------+\n");
          printf("| DISC NAME| %s\n",temphold->cdname);
          printf("+----------+\n");
          printf("| TYPE    | ");
          switch(temphold->disktype)
            {
            case 1: printf("CD");
            break;
            case 2: printf("DVD");
            break;
            case 3: printf("Data CD");
            break;
            case 4: printf("Data DVD");
            break;
            default: printf("UNDEFINED");
            break;
            }

          printf("\n+----------+\n");
          printf("| CONDITION| ");
          if (temphold->condition == 1)
            printf("BAD");
          else
            printf("GOOD");
         
          printf("\n+----------+\n");
          printf("| LISTFILE | ");
          printf("Note at this time listfiles are disabled");
          printf("\n+----------+\n");
          fclose(basefile);
          free(temphold);
          return (looper * sizeof(DATANODE));
        }
  looper++;
   
  }while(feof(basefile) == 0);

  printf("End of file reached record not found\n");
  fclose(basefile);
  free(temphold);
  return 0;
  }
  fclose(basefile);
  free(temphold);
  return 1;
}

[

Could you help to review whether I have understood your points correctly in the sample. The addcd function will add a new node to the beginning of the database file if option is 1, and it will replace the exact node if option is 2, and listfile function will enumerate all the records in a file and print matched one.

My understanding correct?


regards,
George

ta0kira 09-29-2006 08:29 AM

Quote:

Originally Posted by George2
3. What do you mean structures allocate memory? Structures are just static things, how could they allocate memory? :-)

Example:
Code:

struct Dynamic
{
        Dynamic() : Array(new char[128]) {}

        ~Dynamic() { delete Array; }

private:
        char *Array;
};

If you aren't going to use VM, I think a database is the way to go. To implement your objects, have an 'export' type function which writes the value of every data member to the DB, and an 'import' type function (and maybe a constructor) which reads values from the DB and inserts them into the structure. If you're using dynamic memory (like I've shown above), all you need to do is copy the value of what that memory contains rather than the pointer itself. If you have pointers to other structures and you know FOR SURE they will be there still when the structures are reloaded, you can probably just copy the pointers. I think allowing for pointers to structures that might not be there later would be more work that it's worth, however.
ta0kira


All times are GMT -5. The time now is 09:10 AM.