Flume 史上最全面的大数据学习第十篇（六)Flume完结篇

agent 01

a1.sources = r1 a1.channels = c1 a1.sinks = k1 a1.sources.r1.type = TAILDIR a1.sources.r1.filegroups = f1 a1.sources.r1.filegroups.f1 = /root/logs01/.*log.* a1.channels.c1.type = memory a1.sinks.k1.type = avro a1.sinks.k1.hostname = HadoopNode00 a1.sinks.k1.port = 8888 a1.sinks.k1.channel = c1 a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent001.conf --name a1 -Dflume.root.logger=INFO,console

agent 02

a1.sources = r1 a1.channels = c1 a1.sinks = k1 a1.sources.r1.type = TAILDIR a1.sources.r1.filegroups = f1 a1.sources.r1.filegroups.f1 = /root/logs02/.*log.* a1.channels.c1.type = memory a1.sinks.k1.type = avro a1.sinks.k1.hostname = HadoopNode00 a1.sinks.k1.port = 8888 a1.sinks.k1.channel = c1 a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent002.conf --name a1 -Dflume.root.logger=INFO,console

agent 03

a1.sources = r1 a1.channels = c1 a1.sinks = k1 a1.sources.r1.type = TAILDIR a1.sources.r1.filegroups = f1 a1.sources.r1.filegroups.f1 = /root/logs03/.*log.* a1.channels.c1.type = memory a1.sinks.k1.type = avro a1.sinks.k1.hostname = HadoopNode00 a1.sinks.k1.port = 8888 a1.sinks.k1.channel = c1 a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf --name a1 -Dflume.root.logger=INFO,console

agent 04

在avro类型的source中 有一个属性叫做 threads ，在单节点上的测试结果表现为不用设置，即可有多个avroclint可以连接

但是在多节点上使用不同的主机名（域名|ip）进行通讯，实测需要加上threads 属性，threads 的个数由一级agent的个数决定

a2.sources = r1 a2.channels = c1 a2.sinks = k1 a2.sources.r1.type = avro a2.sources.r1.bind = HadoopNode00 a2.sources.r1.port = 8888 a2.channels.c1.type = memory a2.sinks.k1.type =logger a2.sinks.k1.channel = c1 a2.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console

.channels.c1.type = memory

a1.sinks.k1.type = avro a1.sinks.k1.hostname = HadoopNode00 a1.sinks.k1.port = 8888

a1.sinks.k1.channel = c1 a1.sources.r1.channels = c1

> ./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf --name a1 -Dflume.root.logger=INFO,console ### agent 04 > 在avro类型的source中 有一个属性叫做 threads ，在单节点上的测试结果表现为不用设置，即可有多个avroclint可以连接 > > 但是在多节点上使用不同的主机名（域名|ip）进行通讯，实测需要加上threads 属性，threads 的个数由一级agent的个数决定 ~~~properties a2.sources = r1 a2.channels = c1 a2.sinks = k1 a2.sources.r1.type = avro a2.sources.r1.bind = HadoopNode00 a2.sources.r1.port = 8888 a2.channels.c1.type = memory a2.sinks.k1.type =logger a2.sinks.k1.channel = c1 a2.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console

献给每一个正在努力的我们，就算在忙，也要注意休息和饮食哦！我就是我，一个在互联网跌跌撞撞，摸爬滚打的热忱，给个三连吧~ 还有就是不要只看，多动手才行！努力呀！