Hadoop单机安装

    技术2023-09-01  118

    参考文章链接 core-site.xml:

    <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/Hadoop/data/hadoopdata</value> </property> </configuration>

    hdfs-site.xml:

    <configuration> <property> <name>dfs.namenode.name.dir</name> <value>/home/Hadoop/data/hadoopdata/name</value> <description>为了保证元数据的安全一般配置多个不同目录</description> </property> <property> <name>dfs.datanode.data.dir</name> <value>/home/Hadoop/data/hadoopdata/data</value> <description>datanode 的数据存储目录</description> </property> <property> <name>dfs.replication</name> <value>1</value> <description>HDFS 的数据块的副本存储个数, 默认是3</description> </property> <property> <name>dfs.secondary.http.address</name> <value>localhost:50090</value> <description>secondarynamenode 运行节点的信息,和 namenode 不同节点</description> </property> </configuration>

    mapred-site.xml:

    <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>

    yarn-site.xml:

    <configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.resourcemanager.hostname</name> <value>localhost</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> <description>YARN 集群为 MapReduce 程序提供的 shuffle 服务</description> </property> </configuration>

    报错和解决办法: WARN hdfs . DFSUtil: Namenode for null remains unresolved for ID null. Check your hdfs-site.xmL file to ensure namenodes are configured properly. 原因: 未配置SSH免密: 解决办法指令:

    ssh-keygen ssh-copy-id localhost

    故以上配置localhost。

    注意Tips:用ftp连接别用记事本打开修改,否则编码为gbk,导致下图错误: Caused by: org . apache . xerces . impl. io. Mal formedByteSequenceException: Invalid byte 1 of l-byte UTF-8 sequence . 启动HDFS和YARN:

    start-dfs.sh start-yarn.sh

    验证是否启动成功:

    JPS

    访问(我虚拟机ip为:10.9.119.128): http://10.9.119.128:50070/explorer.html#/

    Processed: 0.009, SQL: 9