LogAnalysis
[logs] How do you cull through serial console logs? Mar 07 2008 09:38PM
Zonker Harris (consoleteam gmail com) (2 replies)
AW: [logs] How do you cull through serial console logs? Mar 10 2008 06:48AM
christian folini post ch
Re: [logs] How do you cull through serial console logs? Mar 08 2008 04:51AM
Michael Kinsley (michael kinsley sensage com) (2 replies)
Re: [logs] How do you cull through serial console logs? Mar 10 2008 01:31PM
Ulisses Reina Montenegro de Albuquerque (ulisses tempest com br)
I love Perl and its powerful regular expression engine, but whenever
trying to use multiple regexps in big datasets remember to make sure
your expressions are as tight as possible. In particular:

http://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlre.pod#Backtracking_
_

Even though the map {..} is linear regarding the size of the regular
expression array, applying a regular expression to a string can degrade
to exponential time (which can be nasty for long log lines) -- from the
above URL:

"WARNING: Particularly complicated regular expressions can take
exponential time to solve because of the immense number of possible ways
they can use backtracking to try for a match. For example, without
internal optimizations done by the regular expression engine, this will
take a painfully long time to run:

'aaaaaaaaaaaa' =~ /((a{0,5}){0,5})*[c]/

And if you used *'s in the internal groups instead of limiting them to 0
through 5 matches, then it would take forever--or until you ran out of
stack space. Moreover, these internal optimizations are not always
applicable. For example, if you put {0,5} instead of * on the external
group, no current optimization is applicable, and the match takes a long
time to finish.

A powerful tool for optimizing such beasts is what is known as an
"independent group", which does not backtrack (see (?>pattern)). Note
also that zero-length look-ahead/look-behind assertions will not
backtrack to make the tail match, since they are in "logical" context:
only whether they match is considered relevant. For an example where
side-effects of look-ahead might have influenced the following match,
see (?>pattern)."

On Fri, 2008-03-07 at 20:51 -0800, Michael Kinsley wrote:
> You can use perl's IO::Multiplex module to watch all those files with
> non-blocking IO.
>
>
> Courtesy of the Perl Cookbook and a little extra map{} from me:
>
>
> use IO::Multiplex;
>
>
> my $regex_array = [
> qr/Pattern1/,
>
> qr/Pattern2/,
>
> qr/Pattern..n/
>
>
>
> ];
>
>
> $mux = IO::Multiplex->new( );
> $mux->add($FH1);
> $mux->add($FH2); # ... and so on for all the filehandles to manage
> $mux->set_callback_object(_ _PACKAGE_ _); # or an object
> $mux->Loop( );
>
>
> sub mux_input {
> my ($package, $mux, $fh, $input) = @_;
>
> map { $line =~ m/$_/ and print "Matched Line of Interest: $line
> \n" } @{$regex_array};
> }
>
>
>
> Michael Kinsley
>
>
> Consulting Engineer
>
>
> SenSage, Inc.
> 55 Hawthorne Street Ste. 700
> San Francisco, CA 94105 USA
>
>
> email: michael.kinsley (at) sensage (dot) com [email concealed]
> mobile: +1.415.465.0106
> fax: +1.415.371.1385
>
>
>
>
> On Mar 7, 2008, at 1:38 PM, Zonker Harris wrote:
>
> > I'm using Conserver, which makes reverse-TCP connections to console
> > server serial ports, so I can manage my hosts and net gear. This
> > results in an ASCII text file for each device.
> >
> >
> > What tool(s) can I use to watch all (500+) files for 'interesting'
> > strings, like malloc errors, failed logins, net connection/port/link
> > failures?
> >
> >
> > I've found log watcher, but it is a one- file-at-a-time deal. I'd
> > rather not reinvent the wheel if there is a good answer out there.
> > I'm hoping to get to RSA con this year, and perhaps find other
> > pointers to share.
> >
> >
> > Thank you for any tips,
> >
> >
> > =Z=
> >
> >
> > http://www.conserved.com/consoles/
> > http://consoleteam.blogspot.com/
> >
> >
> > _______________________________________________
> > LogAnalysis mailing list
> > LogAnalysis (at) loganalysis (dot) org [email concealed]
> > http://www.loganalysis.org/mailman/listinfo/loganalysis
>
>
> _______________________________________________
> LogAnalysis mailing list
> LogAnalysis (at) loganalysis (dot) org [email concealed]
> http://www.loganalysis.org/mailman/listinfo/loganalysis
--
Ulisses Reina Montenegro de Albuquerque
AB34 A154 28C1 84A8 676E 202F 264B 117F 88BA B1A5
Matrix Development/Tempest Security Intelligence
http://www.tempest.com.br/

_______________________________________________
LogAnalysis mailing list
LogAnalysis (at) loganalysis (dot) org [email concealed]
http://www.loganalysis.org/mailman/listinfo/loganalysis

[ reply ]
RE: [logs] How do you cull through serial console logs? Mar 10 2008 08:21AM
Rainer Gerhards (rgerhards hq adiscon com) (1 replies)
RE: [logs] How do you cull through serial console logs? Mar 18 2008 06:24PM
Clayton Dukes (cdukes) (cdukes cisco com) (1 replies)
RE: [logs] How do you cull through serial console logs? Mar 19 2008 05:41PM
Balazs Scheidler (bazsi balabit hu) (1 replies)
RE: [logs] How do you cull through serial console logs? Apr 03 2008 03:00AM
Clayton Dukes (cdukes) (cdukes cisco com)


 

Privacy Statement
Copyright 2010, SecurityFocus