OpenForum RSS: awk 'out of memory in array'

awk 'out of memory in array' (AD)

Tue, 19 Feb 2008 05:35:10 GMT

>�� . �� . �� , ��
>��?

�� .
�� 900 ��, 16 �� .

awk 'out of memory in array' (angra)

Mon, 18 Feb 2008 19:27:44 GMT

�� . �� . �� , �� ?

awk 'out of memory in array' (AD)

Mon, 18 Feb 2008 18:09:20 GMT

� �� perl, �� angra �� (�� , �� , � �� :) ).
900 �� 3-4, �� 1,6 ��. �� 180 ��.
�� ;)
�� , �� , �� , �� netflow �� .

awk 'out of memory in array' (angra)

Mon, 18 Feb 2008 13:58:09 GMT

�� , � ��
�� . ��
#!/usr/bin/perl
use strict;
my %sums; #�� , �� ip, �� [�� , �� ]
while (<>) { #��
my @vals = split;#��
my $k="$vals[0] $vals[1]"; #�� ip
$sums{$k}->[0] += $vals[2]; #��
$sums{$k}->[1] += $vals[3]; #��
}
print "Results:\n";
print "$_: $sums{$_}->[0] $sums{$_}->[1]\n" foreach (keys %sums) #�� , �� sort �� keys

��
$ ./a.pl
1.1.1.1 1.1.1.2 10 15
1.1.1.1 1.1.1.3 10 15
1.1.1.1 1.1.1.2 10 15
1.1.1.2 1.1.1.3 20 20
Results:
1.1.1.2 1.1.1.3: 20 20
1.1.1.1 1.1.1.3: 10 15
1.1.1.1 1.1.1.2: 20 30

��

awk 'out of memory in array' (ShyLion)

Mon, 18 Feb 2008 09:57:59 GMT

>>�� awk ��, ��
>>�� .
>
>��, �� "��", �� .
>�� sort-� :) � �� awk-�� _��_ ��
>��.
>�� ݣ �� "��"-�� ... ��, �� .
>

��, �� , � NetFlow, � flow-tools'�� :)

awk 'out of memory in array' (pavel_simple)

Mon, 18 Feb 2008 09:36:57 GMT

>�� , �� . �� , ��
>��.

"�� " (C) ��

Linux XXXX 2.6.22-3-amd64 #1 SMP Tue Feb 12 09:22:35 UTC 2008 x86_64 GNU/Linux

mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

compiled limits:
max NF 32767
sprintf buffer 2040

ls -lah aaa.log
-rw-r--r-- 1 pavel pavel 2,2G 2008-02-18 14:23 aaa.log

time cat aaa.log | grep -v "Accounting"|grep -v "Source"| awk '{s1[$1"\t"$2]+=$3;s2[$1"\t"$2]+=$4;} END {for(k in s1){print k,s1[k],s2[k]}}'>aaa.done.log

real2m11.159s
user2m3.672s
sys0m14.169s

�� , �� aaa.log �� (�� ) �� -�� 100M

awk 'out of memory in array' (Andrey Mitrofanov)

Mon, 18 Feb 2008 08:43:15 GMT

>�� awk ��, ��
>�� .

��, �� "��", �� .
�� sort-� :) � �� awk-�� _��_ �� .
�� ݣ �� "��"-�� ... ��, �� .

awk 'out of memory in array' (AD)

Mon, 18 Feb 2008 03:42:56 GMT

>>�� , �� perl �� .
>
>�� . ��
>�� : (�� ) * (��
>��). �� 20 ��
>�� .
>�� 30+ ��
>�� , � �� .
>

� perl �� , �� :)

awk 'out of memory in array' (AD)

Mon, 18 Feb 2008 03:37:40 GMT

>� ��?
>awk '{if ( NF==4 ){ s1[$1"-"$2]+=$3;s2[$1"-"$2]+=$4;}} END {for(k in s1){print k,s1[k],s2[k]}}'
>�� NF'� �� grep'��

�� , �� . �� , �� .