165 lines
No EOL
13 KiB
HTML
165 lines
No EOL
13 KiB
HTML
<!DOCTYPE html>
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="tool" content="leanote-desktop-app">
|
||
<title>Hadoop集群高可靠安装</title>
|
||
<style>
|
||
|
||
*{font-family:"lucida grande","lucida sans unicode",lucida,helvetica,"Hiragino Sans GB","Microsoft YaHei","WenQuanYi Micro Hei",sans-serif;}
|
||
|
||
body {
|
||
margin: 0;
|
||
}
|
||
|
||
/*公用文字样式*/
|
||
h1{font-size:30px}h2{font-size:24px}h3{font-size:18px}h4{font-size:14px}
|
||
.note-container{
|
||
width:850px;
|
||
margin:auto;
|
||
padding: 10px 20px;
|
||
box-shadow: 1px 1px 10px #eee;
|
||
}
|
||
#title {
|
||
margin: 0;
|
||
}
|
||
table {
|
||
margin-bottom: 16px;
|
||
border-collapse: collapse;
|
||
}
|
||
table th, table td {
|
||
padding: 6px 13px;
|
||
border: 1px solid #ddd;
|
||
}
|
||
table th {
|
||
font-weight: bold;
|
||
}
|
||
|
||
table tr {
|
||
background-color: none;
|
||
border-top: 1px solid #ccc;
|
||
}
|
||
table tr:nth-child(2n) {
|
||
background-color: rgb(247, 247, 249);
|
||
}
|
||
.mce-item-table, .mce-item-table td, .mce-item-table th, .mce-item-table caption {
|
||
border: 1px solid #ddd;
|
||
border-collapse: collapse;
|
||
padding: 6px 13px;
|
||
}
|
||
blockquote {
|
||
border-left-width:10px;
|
||
background-color:rgba(128,128,128,0.05);
|
||
border-top-right-radius:5px;
|
||
border-bottom-right-radius:5px;
|
||
padding:15px 20px;
|
||
border-left:5px solid rgba(128,128,128,0.075);
|
||
}
|
||
blockquote p {
|
||
margin-bottom:1.1em;
|
||
font-size:1em;
|
||
line-height:1.45
|
||
}
|
||
blockquote ul:last-child,blockquote ol:last-child {
|
||
margin-bottom:0
|
||
}
|
||
pre {
|
||
padding: 18px;
|
||
background-color: #f7f7f9;
|
||
border: 1px solid #e1e1e8;
|
||
border-radius: 3px;
|
||
display: block;
|
||
}
|
||
code {
|
||
padding: 2px 4px;
|
||
font-size: 90%;
|
||
color: #c7254e;
|
||
white-space: nowrap;
|
||
background-color: #f9f2f4;
|
||
border-radius: 4px;
|
||
}
|
||
.footnote {
|
||
vertical-align: top;
|
||
position: relative;
|
||
top: -0.5em;
|
||
font-size: .8em;
|
||
}
|
||
|
||
hr {
|
||
margin:2em 0
|
||
}
|
||
img {
|
||
max-width:100%
|
||
}
|
||
pre {
|
||
word-break:break-word
|
||
}
|
||
p,pre,pre.prettyprint,blockquote {
|
||
margin:0 0 1.1em
|
||
}
|
||
hr {
|
||
margin:2em 0
|
||
}
|
||
img {
|
||
max-width:100%
|
||
}
|
||
.sequence-diagram,.flow-chart {
|
||
text-align:center;
|
||
margin-bottom:1.1em
|
||
}
|
||
.sequence-diagram text,.flow-chart text {
|
||
font-size:15px !important;
|
||
font-family:"Source Sans Pro",sans-serif !important
|
||
}
|
||
.sequence-diagram [fill="#ffffff"],.flow-chart [fill="#ffffff"] {
|
||
fill:#f6f6f6
|
||
}
|
||
.sequence-diagram [stroke="#000000"],.flow-chart [stroke="#000000"] {
|
||
stroke:#3f3f3f
|
||
}
|
||
.sequence-diagram text[stroke="#000000"],.flow-chart text[stroke="#000000"] {
|
||
stroke:none
|
||
}
|
||
.sequence-diagram [fill="#000"],.flow-chart [fill="#000"],.sequence-diagram [fill="#000000"],.flow-chart [fill="#000000"],.sequence-diagram [fill="black"],.flow-chart [fill="black"] {
|
||
fill:#3f3f3f
|
||
}
|
||
ul,ol {
|
||
margin-bottom:1.1em
|
||
}
|
||
ul ul,ol ul,ul ol,ol ol {
|
||
margin-bottom:1.1em
|
||
}
|
||
kbd {
|
||
padding:.1em .6em;
|
||
border:1px solid rgba(63,63,63,0.25);
|
||
-webkit-box-shadow:0 1px 0 rgba(63,63,63,0.25);
|
||
box-shadow:0 1px 0 rgba(63,63,63,0.25);
|
||
font-size:.7em;
|
||
font-family:sans-serif;
|
||
background-color:#fff;
|
||
color:#333;
|
||
border-radius:3px;
|
||
display:inline-block;
|
||
margin:0 .1em;
|
||
white-space:nowrap
|
||
}
|
||
.toc ul {
|
||
list-style-type:none;
|
||
margin-bottom:15px
|
||
}
|
||
</style>
|
||
<!-- 该css供自定义样式 -->
|
||
<link href="../leanote-html.css" rel="stylesheet">
|
||
</head>
|
||
|
||
<body>
|
||
|
||
<div class="note-container">
|
||
<h1 class="title" id="leanote-title">Hadoop集群高可靠安装</h1>
|
||
<div class="content-html" id="leanote-content"><div><span style="font-family: 仿宋;" data-mce-style="font-family: 仿宋;"><strong>拓扑图</strong></span></div><div><img id="__LEANOTE_D_IMG_1479696647444" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000df" alt="" data-media-type="image" data-attr-org-src-id="00CFC75C23A04681B4B49888305179D9" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/949c1f67473c4817b038a41e14c372cd/clipboard.png" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000df"><span style="font-family: 仿宋;" data-mce-style="font-family: 仿宋;"><strong><br></strong></span><ol><li><strong>下载<span lang="EN-US">Hadoop</span></strong></li></ol><p><a><span lang="EN-US">wget </span></a><a href="http://apache.fayea.com/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz" data-mce-href="http://apache.fayea.com/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz"><span lang="EN-US">http://apache.fayea.com/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz</span></a></p><ol start="2"><li><strong>解压缩</strong></li></ol><p><span lang="EN-US">tar –zxvf hadoop-2.6.0.tar.gz</span></p><ol start="3"><li><strong>重命名</strong></li></ol><p><span lang="EN-US">mv hadoop-2.6.0 hadoop</span></p><ol start="4"><li><strong>设置环境变量</strong></li></ol><p><a><span lang="EN-US">export HADOOP_HOME=/usr/hadoop</span></a></p><p><span lang="EN-US">export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin</span></p><p><span lang="EN-US">export CLASSPATH=$CLASSPATH:$HADOOP_HOME/lib</span></p><ol start="5"><li><strong>修改配置文件</strong></li></ol><p><span lang="EN-US">Hadoop</span>的配置文件全部都在<span lang="EN-US">$HADOOP_HOME/etc/hadoop</span>目录下</p><p><span lang="EN-US">hadoop-env.sh</span></p><p>修改<span lang="EN-US">Java</span>环境变量</p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647445" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000da" alt="" width="518" height="21" data-media-type="image" data-attr-org-src-id="6885440539CA4D5B9C87504DB3B7DFEA" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/73d556eae17747f1899ca937e5531a63/lip_image001.png" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000da"></span>添加<span lang="EN-US">Hadoop</span>本地类库</p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647446" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000de" alt="" width="554" height="23" data-media-type="image" data-attr-org-src-id="BEE6D29FF96C4BDC89BEC9EB9E79E837" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/c63ff84a71754febba4d88c76c29e97e/lip_image002.png" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000de"></span></p><p><span lang="EN-US">core-site.xml</span></p><p>添加以下内容</p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647447" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000db" alt="" width="503" height="234" data-media-type="image" data-attr-org-src-id="4D17D19E8C274C19A5A1162600A0D309" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/17c64bcd81ab45cb954567a786501d58/lip_image003.png" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000db"></span></p><p><span lang="EN-US">hdfs-site.xml</span></p><p>添加以下内容</p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647448" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000e0" alt="" width="554" height="472" data-media-type="image" data-attr-org-src-id="9714ECC3FBF449569F1D4301D6C90C76" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/45637101b5d54560a86ea84f6aa34c74/ip_image005.jpeg" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000e0"></span></p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647449" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000dc" alt="" width="554" height="351" data-media-type="image" data-attr-org-src-id="55F5AC462287433C8181B69663CE387B" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/6a59dca09f894e68b3b77a3478f40b1c/ip_image007.jpeg" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000dc"></span></p><p><span lang="EN-US">mapred-site.xml</span></p><p>添加以下内容</p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647450" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000dd" alt="" width="511" height="68" data-media-type="image" data-attr-org-src-id="FF0F82F4B0204B8A9014366C0077CC5B" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/1b2b92956f5a44b6bc8e52e8ef1b70a1/lip_image008.png" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000dd"></span></p><p><span lang="EN-US">yarn-site.xml</span></p><p><span lang="EN-US"><img id="__LEANOTE_D_IMG_1479696647451" src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000e1" alt="" width="553" height="490" data-media-type="image" data-attr-org-src-id="8139698DBC9D4120AF01C077E3B958D4" data-attr-org-img-file="file:///C:/Users/Administrator/AppData/Local/YNote/data/hongxiutianmo@163.com/ab977b91f67241bab84d9a9a812cf9ac/ip_image010.jpeg" data-mce-src="http://192.168.56.4:9000/api/file/getImage?fileId=583464b623eeeb0f8f0000e1"></span></p><p><span lang="EN-US">slaves</span></p><p>添加所有<span lang="EN-US">datanode</span>节点的主机名</p><p><span lang="EN-US">slave1</span></p><p><span lang="EN-US">slave2</span></p><p><span lang="EN-US">slave3</span></p><ol start="6"><li><strong>将安装文件拷贝到其他节点</strong></li></ol><p><a><span lang="EN-US">scp –r hadoop master2:/usr/</span></a></p><p><span lang="EN-US">scp –r hadoop resourcemanager1:/usr/</span></p><p><span lang="EN-US">scp –r hadoop resourcemanager2:/usr/</span></p><p><span lang="EN-US">scp –r hadoop slave1:/usr/</span></p><p><span lang="EN-US">scp –r hadoop slave2:/usr/</span></p><p><span lang="EN-US">scp –r hadoop slave3:/usr/</span></p><ol start="7"><li><strong>启动<span lang="EN-US">Zookeeper</span>集群，在每个<span lang="EN-US">zk</span>节点上启动<span lang="EN-US">Zookeeper</span>服务</strong></li></ol><p><span lang="EN-US">zkServer.sh start</span></p><ol start="8"><li><strong>在每个<span lang="EN-US">datanode</span>节点上启动<span lang="EN-US">journalnode</span></strong></li></ol><p><a><span lang="EN-US">hadoop-daemon.sh start journalnode</span></a></p><ol start="9"><li><strong>格式化<span lang="EN-US">HDFS</span>，在<span lang="EN-US">namenode</span>（<span lang="EN-US">master1</span>）上执行即可，不允许多次格式化，如需重复格式化，需要先删除每个节点的<span lang="EN-US">tmp</span>目录，否则<span lang="EN-US">id</span>不一致，集群是启动不了的</strong></li></ol><p><span lang="EN-US">hdfs namenode -format</span></p><p><strong><span lang="EN-US">10</span></strong><strong>、将<span lang="EN-US">master1</span>上的<span lang="EN-US">tmp</span>目录拷贝到<span lang="EN-US">master2</span></strong></p><p><span lang="EN-US"> <a>scp -r tmp master2:/usr/hadoop/</a></span></p><p><strong><span lang="EN-US">11</span></strong><strong>、格式化<span lang="EN-US">Zookeeper</span>，在<span lang="EN-US">master1</span>上执行即可</strong></p><p><span lang="EN-US"> <a>hdfs zkfc -formatZK</a></span></p><p><strong><span lang="EN-US">12</span></strong><strong>、启动<span lang="EN-US">HDFS</span>，在<span lang="EN-US">master1</span>上执行即可</strong></p><p><span lang="EN-US"> start-dfs.sh</span></p><p><strong><span lang="EN-US">13</span></strong><strong>、启动<span lang="EN-US">yarn</span>，在<span lang="EN-US">resourcemanager1</span>上执行</strong></p><p><span lang="EN-US"> start-yarn.sh</span></p><p><span lang="EN-US"> </span>在<span lang="EN-US">resourcemanager2</span>上执行</p><p><span lang="EN-US"> yarn-daemon.sh start resourcemanager</span></p><p><span lang="EN-US"> </span></p><p>参考文献：</p><p><span lang="EN-US"><a href="http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html" data-mce-href="http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html">http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html</a></span></p><p><span lang="EN-US"><a href="http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html" data-mce-href="http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html">http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html</a></span></p><p><span lang="EN-US"><a href="http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html" data-mce-href="http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html</a></span></p><p><span lang="EN-US"><a href="http://www.aboutyun.com/thread-11909-1-1.html" data-mce-href="http://www.aboutyun.com/thread-11909-1-1.html">http://www.aboutyun.com/thread-11909-1-1.html</a></span></p></div></div>
|
||
</div>
|
||
|
||
<!-- 该js供其它处理 -->
|
||
<script src="../leanote-html.js"></script>
|
||
</body>
|
||
</html> |