walminer是一款PostgreSQL的wal日志的解析工具，它可以依据数据字典解析出产生wal日志中隐含的DML语句。第一个版本的xlogminer工具需要wal为logical级别，而且还有表IDENTITY级别的要求。第二个版本的walminer在易用性上不太友好，无法做到精准解析。walminer2.0是第三个版本，这个版本在上一个版本的基础上增强了易用性，可以做到精准解析，并且做了大量的代码重构以期获得更好的扩展性和对PG13的支持。
功能增强 支持指定LSN范围解析 支持特定事务ID解析 支持精确解析语法 增加快捷解析和化身解析模式 版本支持 支持PG9.5,PG9.6,PG10,PG11,PG12,PG13，与上一个版本不同,walminer2.0的代码完成了合并，使用同一份walminer代码可以对应各个版本的PG的编译安装。
编译安装 编译一：从PG源码编译
将walminer目录放置到编译通过的PG工程的"../contrib/"目录下
进入walminer目录
执行命令
make && make install 编译二：依据安装PG的编译
配置pg的bin路径至环境变量
export PATH=/h2/pg_install/bin:$PATH 进入walminer代码路径
执行编译安装
USE_PGXS=1 MAJORVERSION=12 make #MAJORVERSION支持‘9.5’,‘9.6’,‘10’,‘11’,‘12’,‘13’ USE_PGXS=1 MAJORVERSION=12 make install walminer工具使用的标准步骤 walminer2.0与之前的版本一样，都支持在产生wal日志的数据库(生产库)或者在另外的数据库(测试库)执行walminer解析操作。所不同的是，在测试库解析过程需要首先在生产库生成数据字典，然后拿到测试库执行导入数据字典的操作。本博客主要讲述walminer2.0更新的使用接口，用生产库直接解析的方式进行说明，需要在测试库解析的同学可以去阅读下说明文档。
1. 编译安装 略...(不会的可以阅读说明文档或者邮件咨询我)
2. 创建extension postgres=# create extension walminer; CREATE EXTENSION postgres=# 3. 清理walminer内存空间 postgres=# select walminer_stop(); walminer_stop ------------------ walminer stoped! (1 row)
postgres=# 为什么清理空间放在了最前面呢，如果在同一个session中执行多次解析，且有一次执行失败，有些数据可能会遗留在内存中，影响下一次的解析。因此如果你某一次执行解析失败，最好执行一下这个接口，清理一下遗留的内存数据。
4. 管理要解析的wal日志 添加wal日志段
postgres=# select walminer_wal_add('pg_wal'); walminer_wal_add --------------------- 31 file add success (1 row) -- 或者 postgres=# select walminer_wal_add('/h2/pg_walminer_12/data/pg_wal'); walminer_wal_add --------------------- 31 file add success (1 row) -- 或者 postgres=# select walminer_wal_add('/h2/pg_walminer_12/data/pg_wal/000000010000000000000072'); walminer_wal_add -------------------- 1 file add success (1 row) 查看添加的wal日志段
postgres=# select walminer_wal_list(); walminer_wal_list ----------------------------------------------------------- (/h2/pg_walminer_12/data/pg_wal/000000010000000000000072) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000073) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000074) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000075) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000076) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000077) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000078) (/h2/pg_walminer_12/data/pg_wal/000000010000000000000079) (/h2/pg_walminer_12/data/pg_wal/00000001000000000000007A) ... (31 rows)
postgres=#
移除wal段
postgres=# select walminer_wal_remove('/h2/pg_walminer_12/data/pg_wal/00000001000000000000007A'); walminer_wal_remove ----------------------- 1 file remove success (1 row) 目前仅支持一个文件一个文件的移除。
https://www.imdb.com/list/ls080448278/?mode=desktop https://www.imdb.com/list/ls080448231/?mode=desktop https://www.imdb.com/list/ls080448268/?mode=desktop https://www.imdb.com/list/ls080448241/?mode=desktop https://www.imdb.com/list/ls080448288/?mode=desktop https://www.imdb.com/list/ls080448454/?mode=desktop https://www.imdb.com/list/ls080448419/?mode=desktop https://www.imdb.com/list/ls080448461/?mode=desktop https://www.imdb.com/list/ls080448447/?mode=desktop https://www.imdb.com/list/ls080448499/?mode=desktop https://www.imdb.com/list/ls080448908/?mode=desktop https://www.imdb.com/list/ls080448958/?mode=desktop https://www.imdb.com/list/ls080448936/?mode=desktop https://www.imdb.com/list/ls080448922/?mode=desktop https://www.imdb.com/list/ls080448998/?mode=desktop https://www.imdb.com/list/ls080448803/?mode=desktop https://www.imdb.com/list/ls080448876/?mode=desktop https://www.imdb.com/list/ls080448837/?mode=desktop https://www.imdb.com/list/ls080448827/?mode=desktop https://www.imdb.com/list/ls080448844/?mode=desktop https://www.imdb.com/list/ls080448888/?mode=desktop https://www.imdb.com/list/ls080490055/?mode=desktop https://www.imdb.com/list/ls080490017/?mode=desktop https://www.imdb.com/list/ls080490034/?mode=desktop https://www.imdb.com/list/ls080490024/?mode=desktop https://www.imdb.com/list/ls080490085/?mode=desktop https://www.imdb.com/list/ls080490506/?mode=desktop https://www.imdb.com/list/ls080490577/?mode=desktop https://www.imdb.com/list/ls080490535/?mode=desktop https://www.imdb.com/list/ls080490562/?mode=desktop https://www.imdb.com/list/ls080490548/?mode=desktop https://www.imdb.com/list/ls080490582/?mode=desktop https://www.imdb.com/list/ls080490771/?mode=desktop https://www.imdb.com/list/ls080490729/?mode=desktop https://www.imdb.com/list/ls080490781/?mode=desktop https://www.imdb.com/list/ls080490107/?mode=desktop https://www.imdb.com/list/ls080490153/?mode=desktop https://www.imdb.com/list/ls080490172/?mode=desktop https://www.imdb.com/list/ls080490137/?mode=desktop https://www.imdb.com/list/ls080490166/?mode=desktop https://www.imdb.com/list/ls080490141/?mode=desktop https://www.imdb.com/list/ls080490198/?mode=desktop https://www.imdb.com/list/ls080490302/?mode=desktop https://www.imdb.com/list/ls080490370/?mode=desktop https://www.imdb.com/list/ls080490336/?mode=desktop https://www.imdb.com/list/ls080490368/?mode=desktop https://www.imdb.com/list/ls080490344/?mode=desktop https://www.imdb.com/list/ls080490398/?mode=desktop https://www.imdb.com/list/ls080490602/?mode=desktop https://www.imdb.com/list/ls080490658/?mode=desktop https://www.imdb.com/list/ls080490631/?mode=desktop https://www.imdb.com/list/ls080490620/?mode=desktop https://www.imdb.com/list/ls080490646/?mode=desktop https://www.imdb.com/list/ls080490698/?mode=desktop https://www.imdb.com/list/ls080490251/?mode=desktop https://www.imdb.com/list/ls080490276/?mode=desktop https://www.imdb.com/list/ls080490236/?mode=desktop https://www.imdb.com/list/ls080490221/?mode=desktop https://www.imdb.com/list/ls080490290/?mode=desktop https://www.imdb.com/list/ls080490284/?mode=desktop https://www.imdb.com/list/ls080490457/?mode=desktop https://www.imdb.com/list/ls080490476/?mode=desktop https://www.imdb.com/list/ls080490438/?mode=desktop https://www.imdb.com/list/ls080490425/?mode=desktop https://www.imdb.com/list/ls080490442/?mode=desktop https://www.imdb.com/list/ls080490485/?mode=desktop https://www.imdb.com/list/ls080490958/?mode=desktop https://www.imdb.com/list/ls080490978/?mode=desktop https://www.imdb.com/list/ls080490931/?mode=desktop https://www.imdb.com/list/ls080490927/?mode=desktop https://www.imdb.com/list/ls080490996/?mode=desktop https://www.imdb.com/list/ls080490988/?mode=desktop https://www.imdb.com/list/ls080490853/?mode=desktop https://www.imdb.com/list/ls080490878/?mode=desktop https://www.imdb.com/list/ls080490867/?mode=desktop https://www.imdb.com/list/ls080490828/?mode=desktop https://www.imdb.com/list/ls080490889/?mode=desktop https://www.imdb.com/list/ls080495052/?mode=desktop https://www.imdb.com/list/ls080495012/?mode=desktop https://www.imdb.com/list/ls080495032/?mode=desktop https://www.imdb.com/list/ls080495023/?mode=desktop https://www.imdb.com/list/ls080495092/?mode=desktop https://www.imdb.com/list/ls080495504/?mode=desktop https://www.imdb.com/list/ls080495575/?mode=desktop https://www.imdb.com/list/ls080495537/?mode=desktop https://www.imdb.com/list/ls080495564/?mode=desktop https://www.imdb.com/list/ls080495546/?mode=desktop https://www.imdb.com/list/ls080495598/?mode=desktop https://www.imdb.com/list/ls080495757/?mode=desktop https://www.imdb.com/list/ls080495730/?mode=desktop https://www.imdb.com/list/ls080495725/?mode=desktop https://www.imdb.com/list/ls080495741/?mode=desktop https://www.imdb.com/list/ls080495796/?mode=desktop https://www.imdb.com/list/ls080495783/?mode=desktop https://www.imdb.com/list/ls080495177/?mode=desktop https://www.imdb.com/list/ls080495116/?mode=desktop https://www.imdb.com/list/ls080495160/?mode=desktop https://www.imdb.com/list/ls080495126/?mode=desktop https://www.imdb.com/list/ls080495191/?mode=desktop https://www.imdb.com/list/ls080495300/?mode=desktop
