Install and Configure Apache Doris in Ubuntu Environment

Apache Doris is a high-performance, real-time analytic database based on the MPP architecture and is known for its extreme speed and ease of use. It takes only sub-second response times to return query results under massive amounts of data and can support not only highly concurrent point query scenarios, but also high-throughput complex analytic scenarios. This brief guide will show you how to download the latest stable version of Doris, install and run it on a single node, including creating databases, data tables, importing data and queries, etc.

There are two types of nodes in a Doris cluster: Frontend (FE) and Backend (BE).

FE is mainly responsible for metadata management, cluster management, user request access and query plan analysis.

BE is mainly responsible for data storage and execution of query plans.

FE does not participate in the processing and calculation of user data, so it is a node with low resource consumption. The BE is responsible for all data calculations and task processing, and is a resource-consuming node. Therefore, the resource division and resource restriction schemes introduced in this article are all aimed at BE nodes. Because the FE node consumes relatively low resources and can also be scaled horizontally, there is usually no need to isolate and restrict resources, and the FE node can be shared by all users.

Hardware Requirements

Doris runs on a Linux environment, CentOS 7.x or Ubuntu 16.04 or higher is recommended. Both ext4 and xfs file systems are supported.

Development Test Environment

Production Environment

Configure Prerequisite and install Apache Doris

Step 1: OS Requirements

Set the maximum number of open file descriptors in the system .

Execute the command sudo vi /etc/security/limits.conf and add below two lines.

soft nofile 65536
hard nofile 65536

Execute the below commands to disable the swap partition.

sudo apt -y update 
sudo swapoff -a

Step 2: Install Build Tools

Some of the dependencies are not shipped by default and hence the need to manually install them .

sudo apt install build-essential maven cmake byacc flex automake libtool-bin bison binutils-dev libiberty-dev zip unzip libncurses5-dev curl git ninja-build

Step 3: Install Java

Apache Doris requires JAVA 1.8 and above installed in your system. By default, JAVA is available in Ubuntu repositories. Begin by inspecting your default JAVA version with this command

$ java -version
sudo apt install default-jdk

check the JAVA version installed by this command.

$ java -version

set the JAVA_HOME environment variable. This is to make JAVA available system-wide as well as specify the JAVA location in the system. Begin by checking the installation location of JAVA by this command below.

$ sudo update-alternatives --config java

Configure the Java environment. Edit the file below.sudo vi /etc/environment and add the following line

JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64"

Execute the below command to update environment.

source /etc/environment

Step 4: Install Python runtime

Apache Doris requires Python 2.7 and above. By default, Python is available in Ubuntu repositories. Check the version shipped in your distribution by this command:

python3 --version
#or
python --version

If an older version of python exists in your system, update it with the command below.

$ sudo only-upgrade install python3

To Install Python in Ubuntu from Official Repository, run the following commands.

sudo apt update && sudo apt upgrade -y
sudo apt install python3
sudo apt update

Step 5: Install GCC

GCC refers to the C (and C++) compiler offered by the GNU toolchain of compilers. It is an essential tool to compile Apache Doris in Ubuntu. GCC is available in the Ubuntu repositories as gcc package, but for production projects, it’s good to install the build-essential package.

To install GCC in Ubuntu, issue the command below.

sudo apt install gcc-11 g++-11

Create symbolic links for the GCC

sudo ln -s /usr/bin/g++-11 /usr/bin/g++
sudo ln -s /usr/bin/gcc-11 /usr/bin/gcc

Step 6: Doris installation

Navigate to Apache Doris Official Downloads page and copy the latest binary version link. Else download the below links.

ARM64

X64

wget https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-1.2.7.1-bin-x64.tar.xz

Unzip the tarball running this command.

tar -xf apache-doris-1.2.7.1-bin-x64.tar.xz

Step 7: Configure Apache Doris Front End

Do the following to configure Front End

cd  apache-doris-1.2.7.1-bin-x64/fe
sudo vi conf/fe.conf

Modify the following parameters priority_networks and meta_dir as shown below.

meta_dir = ${DORIS_HOME}/doris-meta
priority_networks = 127.0.0.1/24 

Make your changes accordingly. Replace the IP Address with the IP Address of your server machine. Save your changes and exit.

By default, Apache Doris listens on port 8030 on your firewall. Allow this port and other critical ports through the firewall.

sudo ufw allow 8030/tcp
sudo ufw allow 9020/tcp
sudo ufw allow 9030/tcp
sudo ufw allow 9010/tcp

We now need to start the Apache Doris Front End services. Run the command below inside the apache-doris-1.2.7.1-bin-x64\fe directory.

$ ./bin/start_fe.sh --daemon

You can also navigate to your browser using the URL: http://<server_ip_address>:8030. If your configuration was successful, then you should see the Apache Doris login page.

The default username and password are root and no password. Provide these credentials to login.

Step 8: Configure Apache Doris Back End

Change the directory to apache-doris-1.2.7.1-bin-x64/be created from the extraction above. Using your preferred text editor, edit the default configuration file

cd  apache-doris-1.2.7.1-bin-x64/be
sudo vi conf/be.conf

Modify the following parameters priority_networks and storage_path as shown below.

Note: For SSD disks, add .SSD to the end of the directory; for HDD disks, add .HDD.

JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
storage_root_path=/home/disk1/doris.HDD;/home/disk2/doris.SSD;/home/disk2/doris
priority_networks = 127.0.0.1/24 

Make your changes accordingly. Replace the IP. Address with the IP Address of your server machine. Save your changes and exit.

Allow this port and other critical ports through the firewall.

sudo ufw allow 9060/tcp
sudo ufw allow 8040/tcp
sudo ufw allow 9050/tcp
sudo ufw allow 8060/tcp
sudo ufw allow 8060/tcp

We now need to start the Apache Doris Back End services. Run the command below inside the apache-doris-1.2.7.1-bin-x64\be directory.

$ sudo sysctl -w vm.max_map_count=2000000
$ ulimit -n 65536

$ ./bin/start_be.sh --daemon

Step 9: Add BackEnd nodes in FrontEnd service

You can use mysql-client ( Download Mysql) to connect to FE:

mysql -h fe_host -P query_port -u root

Among them, fe_host is the ip of the node where FE is located; query_port is in fe/conf/fe.conf; the root account is used by default, and there is no password to log in.

Once logged in, execute the following command to add each BE:

ALTER SYSTEM ADD BACKEND "be_host:heartbeat-service_port";

Where be_host is the node ip where BE is located; heartbeat_service_port is in be/conf/be.conf.

Once it is addded, check the Active status using below command in mysql client

SHOW PROC '/backends';

References

• Apache Doris Documentation: https://doris.apache.org/
• Connect Apache Doris as a connector in Bold BI: https://support.boldbi.com/kb/article/14946/how-to-connect-apache-doris-in-bold-bi