Elasticsearch hot&warm架构的简单实现

hot&warm架构，简单说，就是将Elasticsearch集群部署划分为hot数据节点和warm数据节点，适用于time based 索引数据。其中，hot数据节点用于处理不断有文档新写入的索引，warm数据节点用于处理不再有新文档写入，但仍有查询需要的索引。

该架构的实现可以参考官网https://www.elastic.co/guide/en/elasticsearch/reference/7.2/shard-allocation-filtering.html 

具体实现：这里以包含有两个节点的集群进行说明，node-1对应hot节点，node-2对应warm节点

1）使用node.attr标记节点

node-1的elasticsearch.yml配置为：

cluster.name: test node.name: node-1 path.data: /home/elk/elasticsearch-7.2.0/data path.logs: /home/elk/elasticsearch-7.2.0/logs http.port: 9200 network.host: 0.0.0.0 discovery.seed_hosts: ["192.168.52.127"] cluster.initial_master_nodes: ["node-1"] node.attr.type: hot

node-2的elasticsearch.yml配置为：

cluster.name: test node.name: node-2 path.data: /home/elk/elasticsearch-7.2.0/data path.logs: /home/elk/elasticsearch-7.2.0/logs http.port: 9200 network.host: 0.0.0.0 discovery.seed_hosts: ["192.168.52.127"] cluster.initial_master_nodes: ["node-1"] node.attr.type: warm

启动node-1,node-2。在kibana中查看：GET /_cat/nodeattrs?v&h=node,attr,value

node attr value node-2 ml.machine_memory 8201084928 node-2 ml.max_open_jobs 20 node-2 xpack.installed true node-2 type warm node-1 ml.machine_memory 8201076736 node-1 xpack.installed true node-1 ml.max_open_jobs 20 node-1 type hot

2)创建索引到node-1节点，即hot节点

PUT logs-2020.07.05 { "settings": { "number_of_shards": 2, "number_of_replicas": 0, "index.routing.allocation.require.type": "hot" } }

查看索引分布：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index node logs-2020.07.05 node-1 logs-2020.07.05 node-1

3）假设logs-2020.07.05不再有新数据写入，将索引移动到node-2节点，即warm节点

PUT logs-2020.07.05/_settings { "index.routing.allocation.require.type": "warm" }

查看索引分布：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index node logs-2020.07.05 node-2 logs-2020.07.05 node-2