-=> Maurice Kinal wrote to Tony Langdon <=-   
      
    MK> -={ Thursday, 28 April 2016, 01:33:37.560060883 +1000 }=-   
      
    MK> Hey Tony!   
      
    TL> mainly to discuss a project with Maurice.   
      
    MK> Sounds great.  How about trying this just to get us started out;   
      
   Looks like I've just brought an echo to this part of the world, as my uplink   
   didn't have it, but found it quickly from his uplink. :)   
      
    MK> -------------- test raw pkt processing start   
    MK> start_regexp="[0-9]{2} [[:alpha:]]{3} [0-9]{2}   
    MK> [0-9]{2}:[0-9]{2}:[0-9]{2}\x00"   
      
    MK> tr '\n' '\37' < raw.pkt | sed -r "s/${start_regexp}/\n&/g" | \   
    MK>   sed -r -n "/^${start_regexp}/ p" | \   
    MK>   gawk -F'\0' 'BEGIN { OFS = "\0" } { print $1, $2, $3, $4, $5 }' | \   
    MK> temp.msg   
    MK> -------------- test raw pkt processing end   
      
    MK> The start_regexp variable is the ftn datetime stamp and I am using that   
    MK> as the start of the message.  'tr' should replace any fake linefeeds   
    MK> with unit seperators to keep offsets the same.  The first sed call   
    MK> places a linefeed in front of all start_regexp it finds and then the   
    MK> second sed prints all the lines that start with a start_regexp. gawk   
    MK> prints out the resulting null delimited fields.  Finally the last sed   
    MK> step if that information is wanted/needed).  The end result should be a   
    MK> file with each line contatining exactly one message, null delimited   
    MK> fields.  'wc -l < temp.msg' will yield exactly how many messages are in   
    MK> that file.   
      
   It will take me some time (and probably some playing with real data and   
   scripts) to fully catch on.  It takes me a little while to understand the   
   nuances of scripts.   
      
    MK> I think this is a great place to start and I'd appreciate any input   
    MK> about the format of temp.data as well as anything that needs   
    MK> troubleshooting, especially sed which can and does vary across   
    MK> different systems.   
      
   The format is good for delineating messages, but do we want to keep track of   
   any individual components such as headers, kludge lines, etc?  If you want to   
   be able to rapidly process those, you might need to modify the format a bit.    
   Of course, if they can be easuly obtained by grep (as I do with some   
   configuration files), then that's a non issue.  Again, I need to play with some   
   real data. :)   
      
      
      
   ... Kitty litter: created by exploding catnip.   
   --- MultiMail/Win32 v0.49   
    * Origin: Freeway BBS - freeway.apana.org.au (3:633/410)