source는 이벤트를 발생시키는 대상을 정의, channel은 최종 목적지인 sink로 전달하기 위한 임시 저장소라고 보면된다. 하나의 source에서 여러 channel로 보내질 수 있다. 그리고 각 channel에서 자신의 sink로 전달해서 서로 다른 저장소에 저장할 수 있겠다. sink는 채널에서 이벤트를 하나씩 꺼내어 저장소에 저장하는 역할을 한다.
설치 가이드 문서에서 발췌
Sources
Type
Description
Implementation Class
avro
Avro Netty RPC event source. Listens on Avro port and receives events from external Avro streams.
AvroSource
netcat
Netcat style TCP event source. Listens on a given port and turns each line of text into an event.
Allows the Flume 1.x agent to receive events from Flume 0.9.4 agents over thrift rpc.
ThriftLegacySource
Other (custom)
You need to specify the fully-qualified name of the custom source, and provide that class (and its dependent code) in Flume's classpath. You can do this by creating a JAR file to hold the custom code, and placing the JAR in Flume'slib directory.
—
Sinks
Type
Description
Implementation Class
null
/dev/null for Flume - blackhole all events received
NullSink
logger
Log events at INFO level via configured logging subsystem (log4j by default)
LoggerSink
avro
Sink that invokes a pre-defined Avro protocol method for all events it receives (when paired with an avro source, forms tiered collection)
AvroSink
hdfs
Writes all events received to HDFS (with support for rolling, bucketing, HDFS-200 append, and more)
HDFSEventSink
file_roll
RollingFileSink
irc
Takes messages from attached channel and relays those to configured IRC destinations.
IRCSink
Other (custom)
You need to specify the fully-qualified name of the custom sink, and provide that class (and its dependent code) in Flume's classpath. You can do this by creating a JAR file to hold the custom code, and placing the JAR in Flume's lib directory.
—
Channels
Type
Description
Implementation Class
memory
In-memory, fast, non-durable event transport
MemoryChannel
jdbc
JDBC-based, durable event transport (Derby-based)
JDBCChannel
Other (custom)
You need to specify the fully-qualified name of the custom channel, and provide that class (and its dependent code) in Flume's classpath. You can do this by creating a JAR file to hold the custom code, and placing the JAR in Flume's lib directory.
—
예를 들어 source에 netcat, channel에 memory, sink를 logger로 설정한다면 다음과 같다.
foo.sources = netcatSrc
foo.channels = memoryChannel
foo.sinks = loggerSink
foo.sources.netcatSrc.type = netcat
foo.sources.netcatSrc.bind = 0.0.0.0
foo.sources.netcatSrc.port = 3333
foo.sinks.loggerSink.type = logger
foo.sources.netcatSrc.channels = memoryChannel
foo.sinks.loggerSink.channel = memoryChannel
foo.channels.memoryChannel.type = memory
foo.channels.memoryChannel.capacity = 100
source와 sink 모두 중간 channel을 설정해서 channel에 이벤트를 공급하고 (source), 이벤트를 소비하는 역할 (sink)을 하도록 한다.
netcatSrc는 3333 포트를 데이트를 받도록 설정하고 sink는 log4 포멧 (INFO) 으로 출력하도록 한다.
