我为什么会写Impala query profile系列文章呢？
一方面：
在日日顺三期项目上，在跟客户做报表时，发现执行一个sql很慢，只select count(*) 就要花费15分钟左右，如果加上逻辑处理，关联配置表，速度会更慢。
从HUE配置调度，执行时间跑了2小时10分钟13秒，最后我把这个进程kill掉了，我打算一探究竟。
首先，查看一下我处理的这些数据量级：
[rrs-hdp-dn01:25003] > select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != '';
Query: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''
Query submitted at: 2020-07-03 11:58:47 (Coordinator: http://rrs-hdp-dn04:25000)
Query progress can be monitored at: http://rrs-hdp-dn04:25000/query_plan?query_id=2945a77ff619defe:b658730000000000
+----------+
| count(*) |
+----------+
| 6100865 |
+----------+
Fetched 1 row(s) in 848.91s
从查询结果来看，数据了并不算大，610万，用时14分钟8秒。那是什么导致查询这么慢呢？查看一下执行计划。
另一方面：
很多Impala用户不知道如何阅读Impala query profile来了解一个查询背后正在执行的操作，从而在此基础上对查询进行调优以充分发挥查询的性能。因此我想写一篇简单的文章来分享我的经验，并希望它可以对希望了解更多信息的人有所帮助。
这是本系列的第1篇，我将介绍一些Impala query profile的基础知识和查看Profile时特别要注意的内容。
获取Impala query profile
首先，获取Impala query profile有两种方法，最简单的方法是在impala-shell中运行查询后执行“PROFILE”语句，如下所示：后面2篇会详细介绍这个profile
具体内容：
[ip:25003] > profile;
Query Runtime Profile:
Query (id=2945a77ff619defe:b658730000000000):
Summary:
Session ID: 24abae22c723db5:2c0a25a81814e8a8
Session Type: BEESWAX
Start Time: 2020-07-03 11:58:47.317039000
End Time: 2020-07-03 12:12:56.216137000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
User: root
Connected User: root
Delegated User:
Network Address: 10.138.232.87:36237
Default Db: default
Sql Statement: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''
Coordinator: rrs-hdp-dn04:22000
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=0B
Per-Host Resource Estimates: Memory=20.00MB
WARNING: The following tables are missing relevant table and/or column statistics.
bms_ldgdb.ldg_690marketization_ledger
F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
03:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=1 row-size=8B cardinality=1
|
02:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=1 row-size=8B cardinality=1
|
F00:PLAN FRAGMENT [RANDOM] hosts=16 instances=16
Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
01:AGGREGATE
| output: count(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=1 row-size=8B cardinality=1
|
00:SCAN KUDU [bms_ldgdb.ldg_690marketization_ledger t]
predicates: industry_code != ''
kudu predicates: is_delete = 0, ledger_status = 2, check_flag IN (4, -1), module IN (1, 2), account_date >= '2020-06-01'
mem-estimate=0B mem-reservation=0B
tuple-ids=0 row-size=15B cardinality=unavailable
----------------
Estimated Per-Host Mem: 20971520
Tables Missing Stats: bms_ldgdb.ldg_690marketization_ledger
Per Host Min Reservation: rrs-hdp-dn03:22000(0) rrs-hdp-dn04:22000(0) rrs-hdp-dn05:22000(0) rrs-hdp-dn08:22000(0) rrs-hdp-dn11:22000(0) rrs-hdp-dn13:22000(0)
Request Pool: root.default
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
----------------------------------------------------------------------------------------------------------------------------------
03:AGGREGATE 1 316.053us 316.053us 1 1 40.00 KB 10.00 MB FINALIZE
02:EXCHANGE 1 14m8s 14m8s 6 1 0 0 UNPARTITIONED
01:AGGREGATE 6 2.746ms 5.205ms 6 1 67.00 KB 10.00 MB
00:SCAN KUDU 6 2m33s 14m8s 6.10M -1 299.00 KB 0 bms_ldgdb.ldg_690marketization_ledger t
Errors:
Planner Timeline: 21.773ms
- Analysis finished: 1.432ms (1.432ms)
- Equivalence classes computed: 1.478ms (46.487us)
- Single node plan created: 21.005ms (19.526ms)
- Runtime filters computed: 21.020ms (14.599us)
- Distributed plan created: 21.096ms (76.103us)
- Lineage info computed: 21.156ms (59.924us)
- Planning finished: 21.773ms (617.260us)
Query Timeline: 14m8s
- Query submitted: 250.799us (250.799us)
- Planning finished: 24.359ms (24.108ms)
- Submit for admission: 25.375ms (1.016ms)
- Completed admission: 25.565ms (189.521us)
- Ready to start on 6 backends: 26.997ms (1.432ms)
- All 6 execution backends (7 fragment instances) started: 33.022ms (6.024ms)
- Rows available: 14m8s (14m8s)
- First row fetched: 14m8s (695.112ms)
- Unregister query: 14m8s (5.330ms)
- ComputeScanRangeAssignmentTimer: 110.741us
ImpalaServer:
- ClientFetchWaitTimer: 697.276ms
- RowMaterializationTimer: 3.129ms
Execution Profile 2945a77ff619defe:b658730000000000:(Total: 14m8s, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
Backend startup latencies: Count: 6, min / max: 3ms / 5ms, 25th %-ile: 3ms, 50th %-ile: 3ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile: 5ms, 99.9th %-ile: 5ms
Per Node Peak Memory Usage: rrs-hdp-dn03:22000(162.18 KB) rrs-hdp-dn04:22000(383.19 KB) rrs-hdp-dn05:22000(213.18 KB) rrs-hdp-dn08:22000(213.18 KB) rrs-hdp-dn11:22000(213.18 KB) rrs-hdp-dn13:22000(213.18 KB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
Averaged Fragment F01:(Total: 14m8s, non-child: 696.183ms, % non-child: 0.08%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:14m8s max:14m8s mean: 14m8s stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 48.02 KB (49168)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 383.19 KB (392387)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 14m8s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 14m8s
- TotalThreadsSysTime: 1.000ms
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 8 (8)
Fragment Instance Lifecycle Timings:
- ExecTime: 695.216ms
- ExecTreeExecTime: 8.846us
- OpenTime: 14m8s
- ExecTreeOpenTime: 14m8s
- PrepareTime: 40.847ms
- ExecTreePrepareTime: 145.192us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 39.850ms, non-child: 39.850ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 38.755ms
AGGREGATION_NODE (id=3):(Total: 14m8s, non-child: 316.053us, % non-child: 0.00%)
- BuildTime: 15.909us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 40.00 KB (40960)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=2):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)
- ConvertRowBatchTime: 24.499us
- PeakMemoryUsage: 0
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
DataStreamReceiver:
- BytesReceived: 96.00 B (96)
- DeserializeRowBatchTimer: 146.930us
- FirstBatchArrivalWaitTime: 5s982ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Coordinator Fragment F01:
Instance 2945a77ff619defe:b658730000000000 (host=rrs-hdp-dn04:22000):(Total: 14m8s, non-child: 696.183ms, % non-child: 0.08%)
MemoryUsage(16s000ms): 26.75 KB, 40.01 KB, 42.13 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 48.02 KB (49168)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 383.19 KB (392387)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 14m8s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 14m8s
- TotalThreadsSysTime: 1.000ms
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 8 (8)
Fragment Instance Lifecycle Timings:
- ExecTime: 695.216ms
- ExecTreeExecTime: 8.846us
- OpenTime: 14m8s
- ExecTreeOpenTime: 14m8s
- PrepareTime: 40.847ms
- ExecTreePrepareTime: 145.192us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 39.850ms, non-child: 39.850ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 38.755ms
AGGREGATION_NODE (id=3):(Total: 14m8s, non-child: 316.053us, % non-child: 0.00%)
- BuildTime: 15.909us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 40.00 KB (40960)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=2):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)
- ConvertRowBatchTime: 24.499us
- PeakMemoryUsage: 0
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
DataStreamReceiver:
BytesReceived(16s000ms): 29.00 B, 64.00 B, 72.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B
- BytesReceived: 96.00 B (96)
- DeserializeRowBatchTimer: 146.930us
- FirstBatchArrivalWaitTime: 5s982ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 2m33s, non-child: 2.541ms, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:6s025ms max:14m8s mean: 2m33s stddev:5m10s
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 1.98
- BloomFilterBytes: 0
- PeakMemoryUsage: 225.68 KB (231099)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 233.02 KB (238609)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 635.297us
- TotalStorageWaitTime: 2m33s
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 5m7s
- TotalThreadsSysTime: 22.496ms
- TotalThreadsUserTime: 189.971ms
- TotalThreadsVoluntaryContextSwitches: 1.31K (1311)
Fragment Instance Lifecycle Timings:
- ExecTime: 1.013ms
- ExecTreeExecTime: 7.958us
- OpenTime: 2m33s
- ExecTreeOpenTime: 2m33s
- PrepareTime: 41.091ms
- ExecTreePrepareTime: 133.511us
DataStreamSender (dst_id=2):(Total: 461.521us, non-child: 461.521us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 265.91 KB/sec
- OverallThroughput: 34.96 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 15.124us
- TransmitDataRPCTime: 63.344us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 68.504ms, non-child: 68.504ms, % non-child: 100.00%)
- CodegenTime: 693.533us
- CompileTime: 6.554ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 21.630ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 39.283ms
AGGREGATION_NODE (id=1):(Total: 2m33s, non-child: 2.746ms, % non-child: 0.00%)
- BuildTime: 2.726ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 2m33s, non-child: 2m33s, % non-child: 100.00%)
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 151.50 KB (155136)
- RowsRead: 1.02M (1016810)
- RowsReturned: 1.02M (1016810)
- RowsReturnedRate: 88.06 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 1 (1)
- ScannerThreadsTotalWallClockTime: 2m33s
- MaterializeTupleTime(*): 2m33s
- ScannerThreadsSysTime: 17.996ms
- ScannerThreadsUserTime: 145.810ms
- ScannerThreadsVoluntaryContextSwitches: 315 (315)
- TotalKuduScanRoundTrips: 293 (293)
- TotalReadThroughput: 0.00 /sec
Fragment F00:
Instance 2945a77ff619defe:b658730000000005 (host=rrs-hdp-dn04:22000):(Total: 14m8s, non-child: 2.048ms, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 83.61 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB
ThreadUsage(16s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 339.18 KB (347323)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 383.19 KB (392387)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 262.704us
- TotalStorageWaitTime: 14m7s
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 28m16s
- TotalThreadsSysTime: 55.991ms
- TotalThreadsUserTime: 364.945ms
- TotalThreadsVoluntaryContextSwitches: 2.66K (2658)
Fragment Instance Lifecycle Timings:
- ExecTime: 532.246us
- ExecTreeExecTime: 7.588us
- OpenTime: 14m8s
- ExecTreeOpenTime: 14m8s
- PrepareTime: 43.153ms
- ExecTreePrepareTime: 147.736us
DataStreamSender (dst_id=2):(Total: 492.978us, non-child: 492.978us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 225.75 KB/sec
- OverallThroughput: 31.69 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 13.202us
- TransmitDataRPCTime: 69.212us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 70.631ms, non-child: 70.631ms, % non-child: 100.00%)
- CodegenTime: 639.905us
- CompileTime: 6.523ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 21.608ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 41.281ms
AGGREGATION_NODE (id=1):(Total: 14m8s, non-child: 5.205ms, % non-child: 0.00%)
ExecOption: Codegen Enabled
- BuildTime: 3.851ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 299.00 KB (306176)
- RowsRead: 1.02M (1016239)
- RowsReturned: 1.02M (1016239)
- RowsReturnedRate: 1.20 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 1 (1)
- ScannerThreadsTotalWallClockTime: 14m8s
- MaterializeTupleTime(*): 14m8s
- ScannerThreadsSysTime: 49.992ms
- ScannerThreadsUserTime: 313.952ms
- ScannerThreadsVoluntaryContextSwitches: 1.67K (1669)
- TotalKuduScanRoundTrips: 1.56K (1560)
- TotalReadThroughput: 0.00 /sec
Instance 2945a77ff619defe:b658730000000002 (host=rrs-hdp-dn13:22000):(Total: 39s554ms, non-child: 3.704ms, % non-child: 0.01%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1s000ms): 44.18 KB, 73.68 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB
ThreadUsage(1s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 213.18 KB (218299)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 213.18 KB (218299)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 1.901ms
- TotalStorageWaitTime: 39s355ms
- TotalThreadsInvoluntaryContextSwitches: 3 (3)
- TotalThreadsTotalWallClockTime: 1m18s
- TotalThreadsSysTime: 10.998ms
- TotalThreadsUserTime: 183.972ms
- TotalThreadsVoluntaryContextSwitches: 1.09K (1094)
Fragment Instance Lifecycle Timings:
- ExecTime: 2.249ms
- ExecTreeExecTime: 10.310us
- OpenTime: 39s511ms
- ExecTreeOpenTime: 39s480ms
- PrepareTime: 41.037ms
- ExecTreePrepareTime: 139.531us
DataStreamSender (dst_id=2):(Total: 317.413us, non-child: 317.413us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 179.50 KB/sec
- OverallThroughput: 49.23 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 11.384us
- TransmitDataRPCTime: 87.047us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 70.153ms, non-child: 70.153ms, % non-child: 100.00%)
- CodegenTime: 761.527us
- CompileTime: 7.312ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 22.364ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 39.409ms
AGGREGATION_NODE (id=1):(Total: 39s480ms, non-child: 2.999ms, % non-child: 0.01%)
ExecOption: Codegen Enabled
- BuildTime: 2.742ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 39s477ms, non-child: 39s477ms, % non-child: 100.00%)
BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 122.00 KB (124928)
- RowsRead: 1.02M (1016532)
- RowsReturned: 1.02M (1016532)
- RowsReturnedRate: 25.75 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 3 (3)
- ScannerThreadsTotalWallClockTime: 39s478ms
- MaterializeTupleTime(*): 39s474ms
- ScannerThreadsSysTime: 6.998ms
- ScannerThreadsUserTime: 137.979ms
- ScannerThreadsVoluntaryContextSwitches: 100 (100)
- TotalKuduScanRoundTrips: 89 (89)
- TotalReadThroughput: 0.00 /sec
Instance 2945a77ff619defe:b658730000000006 (host=rrs-hdp-dn05:22000):(Total: 10s311ms, non-child: 2.554ms, % non-child: 0.02%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 139.50 KB, 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB
ThreadUsage(500.000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 1.95
- BloomFilterBytes: 0
- PeakMemoryUsage: 213.18 KB (218299)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 213.18 KB (218299)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 451.701us
- TotalStorageWaitTime: 10s123ms
- TotalThreadsInvoluntaryContextSwitches: 2 (2)
- TotalThreadsTotalWallClockTime: 20s504ms
- TotalThreadsSysTime: 23.996ms
- TotalThreadsUserTime: 154.976ms
- TotalThreadsVoluntaryContextSwitches: 1.03K (1034)
Fragment Instance Lifecycle Timings:
- ExecTime: 760.590us
- ExecTreeExecTime: 8.396us
- OpenTime: 10s268ms
- ExecTreeOpenTime: 10s237ms
- PrepareTime: 42.298ms
- ExecTreePrepareTime: 126.204us
DataStreamSender (dst_id=2):(Total: 576.789us, non-child: 576.789us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 202.79 KB/sec
- OverallThroughput: 27.09 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 18.201us
- TransmitDataRPCTime: 77.049us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 70.739ms, non-child: 70.739ms, % non-child: 100.00%)
- CodegenTime: 686.682us
- CompileTime: 6.655ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 22.799ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 40.260ms
AGGREGATION_NODE (id=1):(Total: 10s237ms, non-child: 2.163ms, % non-child: 0.02%)
ExecOption: Codegen Enabled
- BuildTime: 2.634ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 10s235ms, non-child: 10s235ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 122.00 KB (124928)
- RowsRead: 1.02M (1017341)
- RowsReturned: 1.02M (1017341)
- RowsReturnedRate: 99.39 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 10s235ms
- MaterializeTupleTime(*): 10s231ms
- ScannerThreadsSysTime: 20.996ms
- ScannerThreadsUserTime: 107.983ms
- ScannerThreadsVoluntaryContextSwitches: 34 (34)
- TotalKuduScanRoundTrips: 31 (31)
- TotalReadThroughput: 0.00 /sec
Instance 2945a77ff619defe:b658730000000001 (host=rrs-hdp-dn03:22000):(Total: 9s700ms, non-child: 2.187ms, % non-child: 0.02%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 28.00 KB, 44.18 KB, 44.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB
ThreadUsage(500.000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 1.95
- BloomFilterBytes: 0
- PeakMemoryUsage: 162.18 KB (166075)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 162.18 KB (166075)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 403.863us
- TotalStorageWaitTime: 9s524ms
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 19s294ms
- TotalThreadsSysTime: 16.997ms
- TotalThreadsUserTime: 147.977ms
- TotalThreadsVoluntaryContextSwitches: 1.03K (1034)
Fragment Instance Lifecycle Timings:
- ExecTime: 834.047us
- ExecTreeExecTime: 7.554us
- OpenTime: 9s660ms
- ExecTreeOpenTime: 9s634ms
- PrepareTime: 38.618ms
- ExecTreePrepareTime: 109.881us
DataStreamSender (dst_id=2):(Total: 457.379us, non-child: 457.379us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 260.60 KB/sec
- OverallThroughput: 34.16 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 17.881us
- TransmitDataRPCTime: 59.957us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 63.064ms, non-child: 63.064ms, % non-child: 100.00%)
- CodegenTime: 627.375us
- CompileTime: 5.744ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 19.537ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 36.839ms
AGGREGATION_NODE (id=1):(Total: 9s634ms, non-child: 1.772ms, % non-child: 0.02%)
ExecOption: Codegen Enabled
- BuildTime: 2.204ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 9s632ms, non-child: 9s632ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 122.00 KB (124928)
- RowsRead: 1.02M (1017071)
- RowsReturned: 1.02M (1017071)
- RowsReturnedRate: 105.58 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 1 (1)
- ScannerThreadsTotalWallClockTime: 9s632ms
- MaterializeTupleTime(*): 9s628ms
- ScannerThreadsSysTime: 10.998ms
- ScannerThreadsUserTime: 110.983ms
- ScannerThreadsVoluntaryContextSwitches: 33 (33)
- TotalKuduScanRoundTrips: 31 (31)
- TotalReadThroughput: 0.00 /sec
Instance 2945a77ff619defe:b658730000000004 (host=rrs-hdp-dn11:22000):(Total: 8s165ms, non-child: 2.354ms, % non-child: 0.03%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 154.18 KB, 103.18 KB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 213.18 KB (218299)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 213.18 KB (218299)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 385.908us
- TotalStorageWaitTime: 7s980ms
- TotalThreadsInvoluntaryContextSwitches: 2 (2)
- TotalThreadsTotalWallClockTime: 16s215ms
- TotalThreadsSysTime: 12.998ms
- TotalThreadsUserTime: 163.975ms
- TotalThreadsVoluntaryContextSwitches: 1.03K (1030)
Fragment Instance Lifecycle Timings:
- ExecTime: 861.953us
- ExecTreeExecTime: 7.334us
- OpenTime: 8s123ms
- ExecTreeOpenTime: 8s093ms
- PrepareTime: 40.960ms
- ExecTreePrepareTime: 145.679us
DataStreamSender (dst_id=2):(Total: 469.050us, non-child: 469.050us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 328.30 KB/sec
- OverallThroughput: 33.31 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.939us
- TransmitDataRPCTime: 47.593us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 69.192ms, non-child: 69.192ms, % non-child: 100.00%)
- CodegenTime: 761.541us
- CompileTime: 6.639ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 22.322ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 39.278ms
AGGREGATION_NODE (id=1):(Total: 8s093ms, non-child: 3.025ms, % non-child: 0.04%)
ExecOption: Codegen Enabled
- BuildTime: 3.004ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 8s090ms, non-child: 8s090ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 122.00 KB (124928)
- RowsRead: 1.02M (1018748)
- RowsReturned: 1.02M (1018748)
- RowsReturnedRate: 125.92 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 8s091ms
- MaterializeTupleTime(*): 8s085ms
- ScannerThreadsSysTime: 9.998ms
- ScannerThreadsUserTime: 114.982ms
- ScannerThreadsVoluntaryContextSwitches: 31 (31)
- TotalKuduScanRoundTrips: 28 (28)
- TotalReadThroughput: 0.00 /sec
Instance 2945a77ff619defe:b658730000000003 (host=rrs-hdp-dn08:22000):(Total: 6s024ms, non-child: 2.396ms, % non-child: 0.04%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 213.18 KB (218299)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 213.18 KB (218299)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 406.573us
- TotalStorageWaitTime: 5s867ms
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 11s935ms
- TotalThreadsSysTime: 13.997ms
- TotalThreadsUserTime: 123.981ms
- TotalThreadsVoluntaryContextSwitches: 1.02K (1021)
Fragment Instance Lifecycle Timings:
- ExecTime: 839.894us
- ExecTreeExecTime: 6.566us
- OpenTime: 5s982ms
- ExecTreeOpenTime: 5s954ms
- PrepareTime: 40.479ms
- ExecTreePrepareTime: 132.040us
DataStreamSender (dst_id=2):(Total: 455.519us, non-child: 455.519us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 398.54 KB/sec
- OverallThroughput: 34.30 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 15.141us
- TransmitDataRPCTime: 39.206us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 67.243ms, non-child: 67.243ms, % non-child: 100.00%)
- CodegenTime: 684.171us
- CompileTime: 6.448ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.95 MB (2041444)
- NumFunctions: 17 (17)
- NumInstructions: 223 (223)
- OptimizationTime: 21.152ms
- PeakMemoryUsage: 111.50 KB (114176)
- PrepareTime: 38.632ms
AGGREGATION_NODE (id=1):(Total: 5s954ms, non-child: 1.312ms, % non-child: 0.02%)
ExecOption: Codegen Enabled
- BuildTime: 1.922ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 67.00 KB (68608)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
KUDU_SCAN_NODE (id=0):(Total: 5s952ms, non-child: 5s952ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 122.00 KB (124928)
- RowsRead: 1.01M (1014934)
- RowsReturned: 1.01M (1014934)
- RowsReturnedRate: 170.50 K/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 5s952ms
- MaterializeTupleTime(*): 5s948ms
- ScannerThreadsSysTime: 8.998ms
- ScannerThreadsUserTime: 88.986ms
- ScannerThreadsVoluntaryContextSwitches: 25 (25)
- TotalKuduScanRoundTrips: 22 (22)
- TotalReadThroughput: 0.00 /sec
[rrs-hdp-dn01:25003] >
也可以在CDH的Web页面（Cloudera Manager Web UI）上点击CM > Impala > Queries，找到刚刚运行的查询，然后点击Query Details：
接着向下滚动找到“Download Profile”按钮：
由于我只是开发报表，CDH的web页面用户名和密码甲方是不会透露给乙方的，这块就不操作了，也不是本系列的重点。
你也可以到执行查询的Impala Daemon（impalad节点，该节点在Impala集群中为coordinator角色）节点的Web界面来下载query profile信息：
https://{impala-daemon-url}:25000/queries
注：这里的乱码是中文注释，不会影响代码执行。
然后点击对应SQL后的“Details”链接转到“Profile” tab页：
注：由于这里只展现了最近25个操作，所以我操作的目前看不到了。
获取了Profile之后，接下来我们开始剖析Profile。
Profile解析
Query Runtime Profile:
Query (id=2945a77ff619defe:b658730000000000):
Summary:
Session ID: 24abae22c723db5:2c0a25a81814e8a8
Session Type: BEESWAX
Start Time: 2020-07-03 11:58:47.317039000
End Time: 2020-07-03 12:12:56.216137000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
User: root
Connected User: root
Delegated User:
Network Address: 10.138.232.87:36237
Default Db: default
Sql Statement: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''
Coordinator: rrs-hdp-dn04:22000
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
Plan:
----------------
我们将其分为几个部分来介绍，这里面有一些频繁用到的重要信息：
1、查询ID：
Query (id=2945a77ff619defe:b658730000000000):
该ID唯一标识在Impala中运行过的SQL，这对于从Impala Daemon日志中用ID查询相关的信息很有用，只需搜索此查询ID，就可以了解SQL运行细节和相关的错误信息。
2、Session类型：
Session Type: BEESWAX
这可以告诉我们连接来自哪里。BEESWAX意味着查询是从impala-shell客户端运行的。如果从Hue运行，类型将是HIVESERVER2，因为Hue是通过HIVESERVER2 thrift连接的。
3、查询起止时间：
Start Time: 2020-07-03 11:58:47.317039000
End Time: 2020-07-03 12:12:56.216137000
这可以告诉我们查询运行了多长时间，此时间包括会话空闲时间（session idle time）。因此，如果你看到在Hue运行一个简单的查询用了几秒钟返回结果，是由于Hue会保持会话打开直到会话关闭或用户运行另一个查询，因此此处的时间可能比正常显示的时间更长。如果通过impala-shell运行，则开始时间和结束时间应与运行时间完全匹配，因为impala-shell在查询完成后会立即关闭查询处理程序。
4、查询状态：
Query Status: OK
这表明查询是否成功完成。OK表示查询成功执行完成。如果有错误，通常会在此处显示，例如，被用户取消（cancelled by user）、会话超时（session timeout）、异常（Exceptions）等。
5、Impala版本：
Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
该信息显示运行查询的Impala版本，如果你发现它与你安装的Impala版本不匹配，则说明未正确安装某些部分。
6、运行查询的用户信息：
User: root
Connected User: root
Delegated User:
该信息表示运行查询的用户。
7、查询所用的数据库：
Default Db: default
该信息表示查询所使用的数据库。
8、查询语句：
Sql Statement: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''
如果你帮助其他人排查问题，则需要此信息，因为需要了解查询的构造方式以及涉及的表，在许多情况下，对查询的简单重写将有助于解决问题或提高查询性能。
9、执行查询的impalad节点：
Coordinator: rrs-hdp-dn04:22000
该信息显示运行查询的Impala daemon服务的主机，通常被称为Coordinator，这有助于我们选择从哪个主机查找INFO、WARNING和ERROR级别的日志。
10、查询选项：
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
该信息显示当前查询使用了哪些查询选项（QUERY OPTIONS），这有助于我们判断是否在用户级别（user level）或池级别（pool level）覆盖了查询选项。一个示例是将Impala Daemon的内存设置为120GB，但小查询仍然失败，并抛出OutOfMemory异常，通过该信息可以帮助我们验证用户是否在会话中将MEM_LIMIT参数设置了太小的值（可能导致OutOfMemory异常）。
总结
本系列的第1篇介绍了查询的摘要（Summary）部分以帮助我们了解Query Profile的基本信息，在本系列的下一部分，将详细介绍查询计划以及Profile的执行摘要（Execution Summary）部分。
