shell自動化配置Hadoop配置文件示例

shell自動化配置Hadoop配置文件示例

#!/bin/bash
read -p 'Please input the directory of hadoop , ex: /usr/hadoop :' hadoop_dir
if [ -d $hadoop_dir ] ; then
  echo 'Yes , this directory exist.'
else
  echo 'Error , this directory not exist.'
  exit 1
fi

if [ -f $hadoop_dir/conf/core-site.xml ];then
  echo "Now config the $hadoop_dir/conf/core-site.xml file."
  read -p 'Please input the ip value of fs.default.name , i.e. hdfs://ip:port :' ip
  i=1
  while [ -z $ip ]
  do
      read -p 'Please input the ip value of fs.default.name , i.e. hdfs://ip:port :' ip
      let i++
      echo $i
      if [ $i -gt 2 ];then
        echo 'You have input three time , done , exit.'
        exit 1
      fi
  done
  if [ $ip = '' ];then
      echo 'The value of ip can not null , exit.'
      exit 1
  fi
  read -p "Please input the port value of fs.default.name , i.e. hafs://$ip:port :" port
  if [ $port = '' ];then
      echo 'The value of port can not null , exit.'
      exit 1
  fi
  read -p 'Please input the dir value of hadoop.tmp.dir :' hadoop_tmp_dir
  if [ $hadoop_tmp_dir = '' ];then
      echo 'The value of hadoop.tmp.dir can not null , exit.'
      exit 1
  else
      if [ ! -d $hadoop_tmp_dir ];then
        echo 'The directory you have input is not exist , we will make it.'
        mkdir -p $hadoop_tmp_dir
      fi
  fi
  tmp_dir=$(echo $hadoop_tmp_dir|sed 's:/:\\/:g')
     
  sed -i "s/ip/$ip/g" $hadoop_dir/conf/core-site.xml
  sed -i "s/port/$port/g" $hadoop_dir/conf/core-site.xml
  sed -i "s/tmp_dir/$tmp_dir/g" $hadoop_dir/conf/core-site.xml
else
  echo "The file $hadoop_dir/core-site.xml doen't exist."
  exit 1
fi
cat $hadoop_dir/conf/core-site.xml
echo 'Config the core-site.xml success !'
echo

關鍵部分分析：

1.  tmp_dir=$(echo $hadoop_tmp_dir|sed 's:/:\\/:g')

我們輸入的$hadoop_tmp_dir是類似：/usr/hadoop/tmp這樣的，如果直接寫在sed裡：sed -i "s/tmp_dir/$hadoop_tmp_dir/g" $hadoop_dir/conf/core-site.xml ，這樣會出現錯誤。因為它解析進去會是：sed -i "s/tmp_dir//usr/hadoop/tmp/g" $hadoop_dir/conf/core-site.xml這樣的，顯然不能滿足，這裡解決的方式是：想辦法將/usr/hadoop/tmp的輸入轉換為：\ /usr\ /hadoop\/tmp的形式，即加入轉義字符。sed ' s:/:\\/:g ' 這樣就可以了，然後通過$()，取值賦給變量。

2. sed -i " s/ip/$ip/g " $hadoop_dir/conf/core-site.xml

這裡就是所謂的對文件中字符串的替換，示例文件為：

root@Ubuntu:/home/houqd/hadoop/conf# cat -n core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
 <property>
  <name>fs.default.name</name>
  <value>hdfs://ip:port</value>
 </property>
 <property>
  <name>hadoop.tmp.dir</name>
  <value>tmp_dir</value>
 </property>
</configuration>

3. 注意其它語法：  if [ ] ; then  else  fi          while [] do  ..done        i=0  let i++

更多Hadoop相關信息見Hadoop 專題頁面 http://www.linuxidc.com/topicnews.aspx?tid=13