type
status
date
slug
summary
tags
category
icon
password
文章目录
- * DolphinScheduler2.x 伪分布式部署
- QA
DolphinScheduler2.x 伪分布式部署
DolphinScheduler 部署说明
操作系统| 版本
Red Hat Enterprise Linux| 7.0 及以上
CentOS| 7.0 及以上
Oracle Enterprise Linux| 7.0 及以上
Ubuntu LTS| 16.04 及以上
CPU| 内存| 硬盘类型| 网络| 实例数量
DolphinScheduler 支持多种部署模式，包括单机模式（Standalone）、伪集群模式（Pseudo-
Cluster）、集群模式（Cluster）等。
单机模式（standalone）模式下，所有服务均集中于一个 StandaloneServer 进程中，并且
其中内置了注册中心 Zookeeper 和数据库 H2 。只需配置 JDK 环境， 就可一键启动 DolphinScheduler，快速体验其功能。
伪集群模式（Pseudo-Cluster）是在单台机器部署 DolphinScheduler 各项服务，该模式 下 master、worker、api
server 、logger server 等服务都只在同一台机器上。Zookeeper 和数据 库需单独安装并进行相应配置。
集群模式（Cluster）与伪集群模式的区别就是在多台机器部署 DolphinScheduler 各项服 务，并且 Master、Worker
等服务可配置多个。
第 3 章 DolphinScheduler 集群模式部署
集群模式下，可配置多个 Master 及多个 Worker 。通常可配置 2~3 个 Master ，若干个 Worker 。由于集群资源有限，此处配置一个
Master，一个 Worker ，集群规划如下。
node1| master、worker
|
（1）节点需部署 JDK（1.8+），并配置相关环境变量。
（2）需部署数据库，支持 MySQL（5.7+）或者 PostgreSQL（8.2.15+）。
（3）需部署 Zookeeper（3.4.6+）。
（4）节点需安装进程树分析工具psmisc。
sudo yum install -y psmisc
（1）上传 DolphinScheduler 安装包到 node1 节点的/opt/software 目录
（2）解压安装包到当前目录
注：解压目录并非最终的安装目录
tar -zxvf apache-dolphinscheduler-2.0.5-bin.tar.gz
DolphinScheduler 元数据存储在关系型数据库中，故需创建相应的数据库和用户。
（1）创建数据库
（2）创建用户
_***注：***_
若出现以下错误信息，表明新建用户的密码过于简单。
ERROR 1819 (HY000): Your password does not satisfy the current policy
requirements
可提高密码复杂度或者执行以下命令降低 MySQL 密码强度级别。
（3）赋予用户相应权限
mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO
'dolphinscheduler'@'%'; mysql> flush privileges;
具体命令如下:
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8
DEFAULT COLLATE utf8_general_ci;
CREATE USER 'dolphinscheduler'@'%' IDENTIFIED BY
'Mr^HYe]9cR]D';
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO
'dolphinscheduler'@'%';
FLUSH PRIVILEGES;
修改解压目录下的 conf/config 目录下的 install_config.conf 文件。
vim conf/config/install_config.conf
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---------------------------------------------------------
INSTALL MACHINE
---------------------------------------------------------
A comma separated list of machine hostname or IP would be installed DolphinScheduler,
including master, worker, api, alert. If you want to deploy in pseudo-distributed
mode, just write a pseudo-distributed hostname
Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5"
ips="node1"
Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine
modify it if you use different ssh port
sshPort="22"
A comma separated list of machine hostname or IP would be installed Master server, it
must be a subset of configuration `ips`.
Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2"
masters="node1"
A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a
subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts
Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default"
workers="node1:default"
A comma separated list of machine hostname or IP would be installed Alert server, it
must be a subset of configuration `ips`.
Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3"
alertServer="node1"
A comma separated list of machine hostname or IP would be installed API server, it
must be a subset of configuration `ips`.
Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1"
apiServers="node1"
A comma separated list of machine hostname or IP would be installed Python gateway server, it
must be a subset of configuration `ips`.
Example for hostname: pythonGatewayServers="ds1", Example for IP: pythonGatewayServers="192.168.8.1"
The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists.
Do not set this configuration same as the current path (pwd)
installPath="/opt/module/dolphinscheduler"
The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh`
script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs
to be created by this user
deployUser="root"
The directory to store local data for all machine we config above. Make sure user `deployUser` have permissions to read and write this directory.
dataBasedirPath="/tmp/dolphinscheduler"
---------------------------------------------------------
DolphinScheduler ENV
- 作者:Maynor
- 链接:https://maynor1024.live/article/2dd1f390-6aa9-8115-9365-e946825396f8
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议，转载请注明出处。
