create new log file when log file size is reached to 10mb

sreemannem · 05-10-2017, 03:45 AM

Hi Sir,

I am redirecting console (script) output into log file
I want create new log file when log file size is reached to 10mb
can you please help me to do this.

Right now i am using below commond but file size increasing oftnely so i want creat new log file once existing one reached to 10MB-
$B/datarouter/data-router.jar > $B/datarouter/logs/stdout.log 2>&1 &

pan64 · 05-10-2017, 04:17 AM

you may want to check logrotate

BW-userx · 05-10-2017, 09:34 AM

Code:

userx%slackwhere ⚡ ~ ⚡> fileSize=$(wc -c < ~/setbg)
userx%slackwhere ⚡ ~ ⚡> [[ $fileSize -ge '10' ]] && echo "file is ge: $fileSize" || echo "file is lt: $fileSize"
file is ge: 4250

KEEP IN MIND that this code is not an actual Ternary Operator, nor did I make adjustments for bytes to Mega Bytes ect...

suicidaleggroll · 05-10-2017, 10:14 AM

Logrotate can't rotate logs that are in use, and redirecting shell output will always keep the log file in use. A while back I created a C program to do precisely what you're asking and I use it all the time, I call it logwrapper.

You run your program like normal, but pipe the output to logwrapper instead of redirecting it to a file. The arguments to logwrapper control the naming convention used, the size limit for the log files, the number of previous files to keep, and the buffer size to use (1 will flush the output every byte which is useful for watching the log file in real time or with slow moving output, but it will increase CPU load for high speed logging).

eg:

Code:

./program 2>&1 | ./logwrapper program.log 100000 10 1

To run in the background you need nohup with bash -c, eg:

Code:

nohup bash -c "./program 2>&1 ./logwrapper program.log 100000 10 1" &> /dev/null &

Every time you run it, it will cycle any previous log files through the chain and then start up with the name you gave it. When that file reaches the specified limit, it will cycle it to name.0, 0->1, 1->2, etc. and start logging again at the name you gave it. As a result, the name you give it will always be the most recent log file, name.0 will be the next older one, name.1 will be the next one after that, and so on.

Build it with:

Code:

gcc -o logwrapper logwrapper.c

Code:

#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>                                                                                                                                                    
#include <unistd.h>                                                                                                                                                      
                                                                                                                                                                         
int main(int argc, char **argv)                                                                                                                                          
{                                                                                                                                                                        
  int i;                                                                                                                                                                 
                                                                                                                                                                         
  // Control variables                                                                                                                                                   
  char *filename;                                                                                                                                                        
  char *filename_ext_src, *filename_ext_dest;                                                                                                                            
  int fnameptr_src, fnameptr_dest;                                                                                                                                       
  unsigned long size;                                                                                                                                                    
  unsigned int number;                                                                                                                                                   
                                                                                                                                                                         
  // I/O variables                                                                                                                                                       
  FILE *outputFile;                                                                                                                                                      
  unsigned long byteCounter;                                                                                                                                             
  size_t bytesRead;                                                                                                                                                      
  unsigned long buf_size;                                                                                                                                                
                                                                                                                                                                         
  // Dummy variable for stat                                                                                                                                             
  struct stat buf;                                                                                                                                                       
                                                                                                                                                                         
  // Print the usage                                                                                                                                                     
  if(argc < 4)                                                                                                                                                           
  {                                                                                                                                                                      
    printf("Usage: logwrapper [filename] [size] [number] [buffer_size]\n");                                                                                              
    return 1;                                                                                                                                                            
  }                                                                                                                                                                      
                                                                                                                                                                         
  // Get the command line arguments                                                                                                                                      
  filename = argv[1];                                                                                                                                                    
  size = atoi(argv[2]);                                                                                                                                                  
  number = atoi(argv[3]);                                                                                                                                                
                                                                                                                                                                         
  // Number of bytes to read from stdin before dumping it to the output file                                                                                             
  // Too small and it will load down the CPU during fast output                                                                                                          
  // Too big and the code will introduce a buffering delay                                                                                                               
  if(argc >= 5)                                                                                                                                                          
  {                                                                                                                                                                      
    buf_size = atoi(argv[4]);                                                                                                                                            
  }                                                                                                                                                                      
  else                                                                                                                                                                   
  {                                                                                                                                                                      
    buf_size = 16;                                                                                                                                                       
  }                                                                                                                                                                      
                                                                                                                                                                         
  char buffer[buf_size];                                                                                                                                                 
                                                                                                                                                                         
  // Set up some temporary holders for our extended filenames                                                                                                            
  filename_ext_src = (char*)malloc(strlen(filename)+1+4); // +4 can handle up to 1000 files                                                                              
  filename_ext_dest = (char*)malloc(strlen(filename)+1+4); // +4 can handle up to 1000 files                                                                             
                                                                                                                                                                         
  // Loop backwards from number-1 to 0, moving log backups as necessary (3 to 4, 2 to 3, etc.)                                                                           
  for(i=number-1;i>=0;i--)                                                                                                                                               
  {                                                                                                                                                                      
    // Copy the filename to the src and dest variables                                                                                                                   
    fnameptr_src = sprintf(filename_ext_src,"%s",filename);                                                                                                              
    fnameptr_dest = sprintf(filename_ext_dest,"%s",filename);                                                                                                            
                                                                                                                                                                         
    // Tack on the extension                                                                                                                                             
    if(i>0) fnameptr_src += sprintf(&filename_ext_src[fnameptr_src],".%d",i-1);                                                                                          
    fnameptr_dest += sprintf(&filename_ext_dest[fnameptr_dest],".%d",i);                                                                                                 
                                                                                                                                                                         
    // Check if this src file exists, if so move it to the dest                                                                                                          
    if(stat(filename_ext_src,&buf) == 0) rename(filename_ext_src,filename_ext_dest);                                                                                     
  }                                                                                                                                                                      
                                                                                                                                                                         
  // Initialize our output file                                                                                                                                          
  byteCounter = 0;                                                                                                                                                       
  outputFile = fopen(filename,"w");                                                                                                                                      
  if(outputFile == NULL)                                                                                                                                                 
  {                                                                                                                                                                      
    fprintf(stderr, "ERROR: Unable to open \"%s\" for writing.\n",filename);                                                                                             
    return 1;                                                                                                                                                            
  }                                                                                                                                                                      
                                                                                                                                                                         
  // Loop forever!!! muahahaha                                                                                                                                           
  while(1)                                                                                                                                                               
  {                                                                                                                                                                      
    // Read from stdin                                                                                                                                                   
    bytesRead = fread(buffer, sizeof(char), buf_size, stdin);                                                                                                            
                                                                                                                                                                         
    // fread does a blocking read of buf_size bytes from the source (stdin)                                                                                              
    // if it exits having read 0 bytes, it means stdin has been closed and we should shut down                                                                           
    if(bytesRead == 0) return 0;                                                                                                                                         
                                                                                                                                                                         
    // Write to our output file                                                                                                                                          
    fwrite(buffer, sizeof(char), bytesRead, outputFile);                                                                                                                 
    fflush(outputFile);                                                                                                                                                  
                                                                                                                                                                         
    // Increment our counter                                                                                                                                             
    byteCounter += bytesRead;                                                                                                                                            
                                                                                                                                                                         
    // Check if we've passed the size threshold                                                                                                                          
    if(byteCounter >= size)                                                                                                                                              
    {                                                                                                                                                                    
      // Time to shift our log files                                                                                                                                     
                                                                                                                                                                         
      // Close our current file and reset the counter                                                                                                                    
      fclose(outputFile);                                                                                                                                                
      byteCounter = 0;                                                                                                                                                   
                                                                                                                                                                         
      // Shift the log files like we did before (3 to 4, 2 to 3, etc.)                                                                                                   
      for(i=number-1;i>=0;i--)
      {
        fnameptr_src = sprintf(filename_ext_src,"%s",filename);
        fnameptr_dest = sprintf(filename_ext_dest,"%s",filename);
        if(i>0) fnameptr_src += sprintf(&filename_ext_src[fnameptr_src],".%d",i-1);
        fnameptr_dest += sprintf(&filename_ext_dest[fnameptr_dest],".%d",i);
        
        if(stat(filename_ext_src,&buf) == 0) rename(filename_ext_src,filename_ext_dest);
      }
      
      // Open our new output file
      outputFile = fopen(filename,"w");
      if(outputFile == NULL)
      {
        fprintf(stderr, "ERROR: Unable to open \"%s\" for writing.\n",filename);
        return 1;
      }
    }
  }
  
  // The code will never reach this point, but this is good programming practice anyway
  free(filename_ext_src);
  free(filename_ext_dest);
  
  return 0;
}

MadeInGermany · 05-12-2017, 02:14 PM

Maybe split can do it

Code:

man split

Example

Code:

$B/datarouter/data-router.jar 2>&1 | split -l 100000 - $B/datarouter/logs/stdout.log &