o1o1o1o1o's Blogger: 快速架一個 CI Server

本篇主要目的為建立 Jenkins 環境，關於 Jenkins 設定則另外介紹

Environment Setup

CentOS Linux release 7.3.1611 (Core)
JDK 1.7.0.131
Tomcat 7.0.76
Jenkins 2.48

Linux 作業系統就不多說了，我是安裝在 VM 上
Jenkins 官網中並沒有說明記憶體最低需求，但建議給 16GB (您真是愛開玩笑呢)
我給它 6GB，Jenkins plugins 不要裝太多還夠用

1. Install JDK 1.7

因為產品是以 JDK 1.7 進行開發的，最好保持環境一致

1.1 2017/06/05 更新

Oracle 已不提供 JDK 1.7 的載點，從GOOGLE找到FTP載點下載 jdk-7u80-linux-x64.tar.gz，並手動安裝
安裝 Tomcat7 之後，再安裝 JDK 1.7的話，有可能會被 OpenJDK 蓋過設定。記得RECHECK

tar -xzvf jdk-7u80-linux-x64.tar.gz
sudo mv jdk1.7.0_80/ /usr/lib/jvm/
 
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0_80/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0_80/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0_80/bin/javaws" 1
 
sudo update-alternatives --config java

There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      auto mode
  1            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      manual mode
  2            /usr/lib/jvm/jdk1.7.0_80/bin/java                1         manual mode

Press enter to keep the current choice[*], or type selection number: 2

Java Version

java -version

java version "1.7.0_80" 
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)

1.2 In CentOS 7 (Older way)

# 下載 JDK 1.7
wget \
    --no-cookies \
    --no-check-certificate \
    --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" \
    -O jdk-7u80-linux-x64.rpm \
    http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.rpm
 
# root 權限安裝
rpm -ivh jdk-7u80-linux-x64.rpm
 
# Java will be installed into /usr/java
ll /usr/java/

total 4
lrwxrwxrwx 1 root root   16 Mar  6 10:52 default -> /usr/java/latest
drwxr-xr-x 8 root root 4096 Mar  6 10:52 jdk1.7.0_80
lrwxrwxrwx 1 root root   21 Mar  6 10:52 latest -> /usr/java/jdk1.7.0_80

1.3 In Ubuntu OS (Older way)

參考 https://help.ubuntu.com/community/Java

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer

安裝成功後，路徑應為 /usr/lib/jvm/java-7-oracle

2. Install Tomcat 7

因為產品是使用 Java Servlet ，所以選擇 Tomcat 7

# 下載 Tomcat 7 tar，直接佈署
wget http://ftp.tc.edu.tw/pub/Apache/tomcat/tomcat-7/v7.0.76/bin/apache-tomcat-7.0.76.tar.gz
 
# 解壓縮
tar -xf apache-tomcat-7.0.76.tar.gz
 
# 搬到 /usr/local/tomcat7，需要 root 權限
mv apache-tomcat-7.0.76 /usr/local/tomcat7

當要執行/結束 Tomcat 7 時，可以下指令

# 先移動到 Tomcat7 目錄
cd /usr/local/tomcat7/
 
# Start script
./bin/startup.sh
 
# Shutdown script
./bin/shutdown.sh

2.1 Tomcat Service Script

不過每次都要這麼做，其實也蠻累的
重點是，我還想讓它可以開機時啟動啊

2.1.1 Systemctl command

CentOS 7 使用 systemctl 取代 service 指令

i). Tomcat 7 Environment Setup

Tomcat 建議可以建立 setenv.sh/setenv.bat 來設定環境
在執行 catalina.sh 時，如果 setenv.sh 存在時就會自動執行
預設是不存在，所以手動建立新檔案 /usr/local/tomcat7/bin/setenv.sh
可參考 https://gist.github.com/patmandenver/cadb5f3eb567a439ec01

#
# Cutom Environment Variables for Tomcat
#
############################################
export JAVA_HOME=/usr/java/jdk1.7.0_80/jre
export PATH=${JAVA_HOME}/bin:${PATH}
 
############################################
#
# JAVA_OPTS
#  You could do all this in one export command
# But I am going to be long winded and explain
# Why and add links
#
# Oracle Notes
#  - https://docs.oracle.com/javase/8/docs/technotes/tools/windows/java.html
#  - A good visual
#  https://redstack.wordpress.com/2011/01/06/visualising-garbage-collection-in-the-jvm/
#############################################
 
# -server
#  Select the java HotSpot Server JVM
# The 64-bit version of the JDK support only the Server VM,
# so in that case the option is implicit
# ... so it's redundant to today's world but it make me feel good.
export JAVA_OPTS="-server" 
 
# -Xms/Xmx
#   Xms Sets the initial size of the Heap
#   Xmx sets the Maximum size of the Heap.
#  http://stackoverflow.com/questions/16087153/what-happens-when-we-set-xmx-and-xms-equal-size
#  http://crunchify.com/jvm-tuning-heapsize-stacksize-garbage-collection-fundamental/
export JAVA_OPTS="$JAVA_OPTS -Xms1024M -Xmx1024M" 
 
# -NewSize/MaxNewSize
#  Set the size of the young generation
#  Most newly created objects are made here
#  Objects taht did not become unreachbale and survice the young
# Generation heap are copied to the Old Generation
# See http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection
# https://redstack.wordpress.com/2011/01/06/visualising-garbage-collection-in-the-jvm/
export JAVA_OPTS="$JAVA_OPTS -XX:NewSize=512m -XX:MaxNewSize=512m" 
 
# -PermSize/MaxPermSize
#  Store classes and interned character strings
# http://stackoverflow.com/questions/12114174/what-does-xxmaxpermsize-do
#   Warning!
#  Decprecated in Java 8!!  replace -XX:MetaspaceSize  !!!
export JAVA_OPTS="$JAVA_OPTS -XX:PermSize=256m -XX:MaxPermSize=256m" 
 
# -HeapDumpPath
#   Set the file where the heap dump will write out its error
export JAVA_OPTS="$JAVA_OPTS -XX:HeapDumpPath=/usr/local/tomcat/logs/java_heapdump_pid_%p.log" 
 
# -java.security.egd
#  This one is abit of a debate
# If you don't set this it will use /dev/random on startup
# which can block and make tomcat startup slower.
# But it's technically more secure... but no one has shown
# a way to break the results of urandom which is faster.
#  For more details see.
#   http://www.2uo.de/myths-about-urandom/
export JAVA_OPTS="$JAVA_OPTS -Djava.security.egd=file:/dev/./urandom" 
 
#################################################
# CATALINA_OPTS
# This are basically JAVA_OPTS but only used by tomcat
# and only run on Tomcat start see
#  http://stackoverflow.com/questions/11222365/catalina-opts-vs-java-opts-what-is-the-difference
# for more details
#
################################################
 
# Add tomcat pid
CATALINA_PID="$CATALINA_BASE/tomcat.pid"

ii). Tomcat 7 Systemctl Script

在 /usr/lib/systemd/system 中，新建新檔案 tomcat7.service
由於指定了一般使用者 ciserver，所以 PIDFile path 要在這個使用者有權限的地方
(不然通常是放在 /var/run/)

[Unit]
Description=Apache Tomcat 7
After=syslog.target network.target
 
[Service]
Type=forking
 
PIDFile=/usr/local/tomcat7/tomcat.pid
ExecStart=/usr/local/tomcat7/bin/startup.sh
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
 
User=ciserver
Group=ciserver
 
[Install]
WantedBy=multi-user.target

常用指令 (需 root)

# 當修改檔案內容後，都需要重新 LOAD
systemctl daemon-reload
 
# 執行
systemctl start systemd-tomcat
 
# 停止
systemctl stop systemd-tomcat
 
# 開機時自動執行
systemctl enable systemd-tomcat
 
# 取消開機自動執行
systemctl disable systemd-tomcat

2.1.2 Service command (Older way)

上網找了大大提供的 tomcat service script
下載並修改內容中的名稱與路徑，放到 /etc/init.d/tomcat7

#!/bin/bash
#
# tomcat
#
# chkconfig: 345 96 30
# description:  Start up the Tomcat servlet engine.
#
# processname: java
# pidfile: /var/run/tomcat.pid
#
### BEGIN INIT INFO
# Provides: tomcat
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Should-Start: distcache
# Short-Description: start and stop Apache HTTP Server
# Description: implementation for Servlet 2.5 and JSP 2.1
## END INIT INFO
 
# Source function library.
. /etc/init.d/functions
 
## tomcat installation directory. *[改這裡]
PROCESS_NAME=tomcat
 
## *[改這裡]
CATALINA_HOME="/usr/local/tomcat7/"
 
## run as a diffent user. *[改這裡]
TOMCAT_USER=ciserver
 
##  Path to the pid, runnning info file
pidfile=${PIDFILE-/var/run/${PROCESS_NAME}.pid};
lockfile=${LOCKFILE-/var/lock/subsys/${PROCESS_NAME}};
 
RETVAL=0
 
case "$1" in
 start)
        PID=`pidofproc -p ${pidfile} ${PROCESS_NAME}`
        if [[ (-n ${PID}) && ($PID -gt 0) ]]; then
                logger -s "${PROCESS_NAME}(pid ${PID}) is  already running."
                exit;
        fi
        if [ -f $CATALINA_HOME/bin/startup.sh ];
          then
            logger -s "Starting Tomcat"
            /bin/su -l ${TOMCAT_USER} -c "$CATALINA_HOME/bin/startup.sh -Dprocessname=${PROCESS_NAME}"
            PID=`ps -eaf|grep processname=${PROCESS_NAME}|grep -v grep|awk '{print $2}'`
            RETVAL=$?
            [ $RETVAL = 0 ] && touch ${lockfile}
            [ $RETVAL = 0 ] && echo "${PID}" > ${pidfile}
        fi
        ;;
 stop)
        PID=`pidofproc -p ${pidfile} ${PROCESS_NAME}`
        ## if PID valid run shutdown.sh
        if [[ -z ${PID} ]];then
            logger -s "${PROCESS_NAME} is not running."
            exit;
        fi
 
        if [[ (${PID} -gt 0) && (-f $CATALINA_HOME/bin/shutdown.sh) ]];
          then
            logger -s "Stopping Tomcat"
            /bin/su -l ${TOMCAT_USER} -c "$CATALINA_HOME/bin/shutdown.sh"
            RETVAL=$?
            [ $RETVAL = 0 ] && rm -f ${lockfile}
            [ $RETVAL = 0 ] && rm -f ${pidfile}
        fi
        ;;
 status)
        status -p ${pidfile} ${PROCESS_NAME}
        RETVAL=$?
        ;;
 restart)
         $0 stop
         $0 start
         ;;
version)
        if [ -f $CATALINA_HOME/bin/version.sh ];
          then
            logger -s "Display Tomcat Version"
            /bin/su -l ${TOMCAT_USER} -c "$CATALINA_HOME/bin/version.sh"
            RETVAL=$?
        fi
        ;;
 *)
         echo $"Usage: $0 {start|stop|restart|status|version}"
        exit 1
        ;;
esac
exit $RETVAL

2.2 tomcat-users.xml 設定檔

Setup User Accounts ，讓 Jenkins 把 Web 專案佈署在 Tomcat7 上

cd /usr/local/tomcat7
 
# Edit conf/tomcat-users.xml file and paste inside <tomcat-users></tomcat-users>
<!-- For IDE/Jenkins can manager tomcat -->
<role rolename="manager-script" />
<user username="manager" password="_SECRET_PASSWORD_" roles="manager-script" />

2.3 Enable HTTPS + AuthBind

自己產生一個憑證供使用

cd /usr/local/tomcat7/
mkdir keystore
cd keystore
 
# Generate the keystore
# keytool -genkey -alias tomcat -keyalg RSA -keysize 2048 -keystore tomcat.keystore

Enter keystore password:                   # {password}自行輸入
Re-enter new password: 
What is your first and last name?
  [Unknown]:  Clone Lin
What is the name of your organizational unit?
  [Unknown]:  private
What is the name of your organization?
  [Unknown]:  private
What is the name of your City or Locality?
  [Unknown]:  Taiwan
What is the name of your State or Province?
  [Unknown]:  Clone
What is the two-letter country code for this unit?
  [Unknown]:  tw
Is CN=Clone Lin, OU=private, O=private, L=Taiwan, ST=Clone, C=tw correct?
  [no]:  yes

Enter key password for 
 (RETURN if same as keystore password):  
Re-enter new password:


# ll
total 4
-rw-rw-r-- 1 clone clone 2232 Mar  9 08:44 tomcat.keystore

修改 tomcat server 設定，開啟 SSL/TLS port 443

cd /usr/locat/tomcat7/
 
# Edit conf/server.xml
<Connector port="443" protocol="HTTP/1.1"
          maxThreads="200"
          redirectPort="8443"
          URIEncoding="UTF-8"
          scheme="https" secure="true" SSLEnabled="true"
          keystoreFile="/usr/local/tomcat7/keystore/tomcat.keystore"
          keystorePass="_SECRET_PASSWORD_"
          clientAuth="false" sslProtocol="TLS" />

當 Bind port number < 1024 時，會需要 root 權限。但我又不想讓 tomcat 有 root 權限
這裡使用 AuthBind 使 User 可以成功 Bind port 443

因為 CentOS 沒辦法使用 yum 安裝，所以要下載編譯後安裝
GitHub : https://github.com/tootedom/authbind-centos-rpm

# 下載並編譯安裝
wget http://ftp.debian.org/debian/pool/main/a/authbind/authbind_2.1.1.tar.gz
tar -xf authbind_2.1.1.tar.gz
cd authbind-2.1.1/
make
# root 權限安裝
make install
 
# 開啟 tomcat 將使用的port (這裡的動作都要root 權限)
touch /etc/authbind/byport/443
chmod 500 /etc/authbind/byport/443
chown {username}:{group} /etc/authbind/byport/443
 
# Tomcat 若是 yum 安裝，預設帳號與群組是 “tomcat”
 
# 修改啟動 Tomcat Server 的腳本
# 可以修改 /usr/local/tomcat/bin/catalina.sh or /usr/local/tomcat/bin/startup.sh
# 這裡選擇修改 startup.sh
# 將原本執行的指令註解，新增 authbind --deep。修改後結果如下:
 
#exec "$PRGDIR"/"$EXECUTABLE" start "$@"
exec authbind --deep "$PRGDIR"/"$EXECUTABLE" start "$@"

到這一步，重啟Tomcat Server就可以了

2.4 Iptables rules

預設是擋掉的，自已增加規則
這裡用 8080 port 舉例，443 port 也是用相同做法
(iptables -A means Append; iptables -I means Insert)

# 嚴謹一點的像是
iptables -I INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT -m comment --comment "Tomcat Server port"
 
# 簡單一點的就是
iptables -I INPUT -p tcp --dport 8080 -j ACCEPT
 
# 顯示目前 IPTABLES 規則
iptables-save
 
# 儲存 IPTABLES 規則
# CentOS 已經改用 Firewalld 取代 IPTABLES 了，可以使用下列指令新增規則
# 若不加上 --permanent 表示為 Runtime rule，不會儲存
firewall-cmd --zone=public --add-port=443/tcp --permanent
 
# 重新載入，確認是否成功
firewall-cmd --reload

3. Install Jenkins 2.48

官網 : https://jenkins.io/

有兩種安裝方式，一種是下載 war 檔案，放到 tomcat/webapp 中執行口別一種是使用 RPM 安
裝方式
這裡使用 war 方式安裝

# 選擇 stable 版本
wget http://mirrors.jenkins.io/war-stable/latest/jenkins.war
cp jenkins.war /usr/local/tomcat7/webapps/

打開 Browser，輸入 URL: http://127.0.0.1/jenkins/
第一次安裝 Jenkins 時，會提示輸入安裝密碼
按照提示將密碼貼到對應的輸入框即可

cat /home/clone/.jenkins/secrets/initialAdminPassword

60a9fe50f451436b987979704006b33a

安裝類型有通用安裝 / 自定義安裝

安裝過程較長，成功畫面如下

上圖是測試安裝的截圖，是英文版的
後來選擇 Jenkins stable 版本，安裝後是中文版的

o1o1o1o1o's Blogger

2017年3月30日星期四

快速架一個 CI Server - Jenkins 環境建置

Environment Setup

1. Install JDK 1.7

1.1 2017/06/05 更新

1.2 In CentOS 7 (Older way)

1.3 In Ubuntu OS (Older way)

2. Install Tomcat 7

2.1 Tomcat Service Script

2.1.1 Systemctl command

2.1.2 Service command (Older way)

2.2 tomcat-users.xml 設定檔

2.3 Enable HTTPS + AuthBind

2.4 Iptables rules

3. Install Jenkins 2.48

沒有留言:

張貼留言

2017年3月30日 星期四

快速架一個 CI Server - Jenkins 環境建置

Environment Setup

1. Install JDK 1.7

1.1 2017/06/05 更新

1.2 In CentOS 7 (Older way)

1.3 In Ubuntu OS (Older way)

2. Install Tomcat 7

2.1 Tomcat Service Script

2.1.1 Systemctl command

2.1.2 Service command (Older way)

2.2 tomcat-users.xml 設定檔

2.3 Enable HTTPS + AuthBind

2.4 Iptables rules

3. Install Jenkins 2.48

沒有留言:

張貼留言

2017年3月30日星期四