[Berlin-wireless] Summer of Code idea: measurement in mesh networks

[Alexey Mikhailov]

Hello!

I had seen your idea regarding measurement in mesh networks.
You suggest to port OpenIMP to obtain measurement functionality
on devices. I don't agree that OpenIMP will suit for embedded
Linux platform. If you will look at its implementation, it uses
libpcap for packet capturing, so it wouldn't keep with high
speed links obviously. Moreover, it doesn't support bidirectional
flows (RFC 5103).

I want to propose new efficient metering process (generating flow
information in IPFIX/Netflow format). It is based on using conntrack
subsystem of Linux 2.6 kernel.

Motivation:

1) Speed. Bandwidth of network channels is growing up, we need to have
tools that can challenge in this environment.

2) Using IPFIX. First of all, we can specify flow information elements
we want to export (IPFIX is based on template-based Netflow version 9
format). Another very cool feature is bidirectional flows (RFC 5103).
Bidirectional flow is simple, we just can define flow not as
unidirectional sequence but as bidirectional. This definition is more
essential for connection. Some trivial application of bidirectional
flows are:

 * Separate traffic into ``answered'' and not
 * Getting RTT is very easy (reverse flow begtime - forward flow begtime)
 * Full reconstruction of TCP session
 * With Netflow V5 you can't even support IPV6--flows...

3) IPFIX is very young protocol (RFC 5101, 5102, 5103 comes from 2008), so
there're no much tools for it: libipfix, openIMP, CERT software (libfixbuf,
yaf, silk), VERMONT. Most of them doesn't support bidirectional flows
(e.g. SiLK divides bidirection flow into two unidirectional ones) or have
other problems. I think developing another effective tools is worthwhile.

Little on technical side:

Linux already aggregates packets in flows, it's necessary for NAT and
such. This subsystem is called connection tracking. Obviously,
notion of bidirectional flow is essential there. So we just need
to export those "conntracks" from kernel. (there are some other
problems like passive timeout that i don't touch here)

I had worked on such system and I already got results, though
it's not general enough, as it was developed for our network
environment. It has proven to be very effective.

I don't want to give much technical details beforehand. But what
I propose is:

1) Porting IPFIX probe to OpenWRT platform, allowing device to
export flows in IPFIX format. Also it will include generalization
of this solution.

2) Integrate it with freimap using modified libipfix library.
I had modified it to support bidirectional flows and reduce
memory footprint while porting to Linux kernel (IPFIX
protocol support in my kernel-based probe was originally
based on libipfix as it has some simple and compact
implementation)

Again, I don't want to give many details before as I'm not sure you're
looking for what I propose. So I wait for your reply! Any questions are
welcome, and I can post technical details, proof-of-concept code and
detailed timeline. And excuse me for my poor English.

Thank you in advance,
-- Alexey.