Step by step install MongoDB Sharding Cluster di Ubuntu 18.04, semoga dapat mundah untuk di implementasikan

Berikut design nya

Disini saya menggunakan 8 Virtual Machine, sesuai design diatas.

Langkah pertama

Semua perintah yang ditutorial ini menggunakan akses USER bukan sebagai root.

Rubah semua hostname pada server sesuai tabel diatas (8 Server)

Change Hostname Ubuntu 18.04

sudo hostnamectl set-hostname cfg1

Atur timezone sesuai dengan daerah

sudo timedatectl set-timezone Asia/Jakarta

Merubah Hosts file

Merubah hosts file di semua server

# Config Server
192.168.22.61 cfg1.contoh.com
192.168.22.62 cfg2.contoh.com
192.168.22.63 cfg3.contoh.com

# Shard Server Replica Set (rs0)
192.168.22.81 shrd1.contoh.com
192.168.22.82 shrd2.contoh.com
192.168.22.83 shrd3.contoh.com

# Mongos (Query Router)
192.168.22.60 qrt1.contoh.com
192.168.22.80 qrt2.contoh.com

Menambahkan Repository MongoDB

Ini juga dilakukan di semua server (8 server), kemudian lakukan install MongoDB

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4b7c549a058f8b6b
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list

sudo apt update
sudo apt -y install mongodb-org

Membuat akun admin pada MongoDB

Lakukan di semua server (8 Server)

Start MongoDB

sudo systemctl start mongod.service
sudo systemctl enable mongod.service

Masuk ke shell mongodb

mongo

ketik perintah berikut di mongo shell

use admin

Buat user admin pada mongodb

db.createUser({user: "administrator", pwd: "password", roles:[{role: "root", db: "admin"}]})

ganti “password” dengan password yang lebih aman

Keluar dari mongo shell

quit()

Setting MongoDB Authentication

Masuk ke server cfg1.contoh.com (192.168.22.61)

openssl rand -base64 756 > ~/mongodb_key.pem

Ikuti perintah berikut

sudo cp ~/mongodb_key.pem /var/lib/mongodb/
sudo chown -R mongodb:mongodb /var/lib/mongodb/mongodb_key.pem
sudo chmod -R 400 /var/lib/mongodb/mongodb_key.pem

Copy file mongodb_key.pem ke semua server

scp ~/mongodb_key.pem user@192.168.22.62:/var/lib/mongodb/

lakukan ke semua server (8 server)

Setting Config Server Replica set

Server cfg1.contoh.com edit file /etc/mongod.conf

storage:
 dbPath: /var/lib/mongodb
 journal:
  enabled: true

systemLog:
 destination: file
 logAppend: true
 path: /var/log/mongodb/mongod.log

net:
 port: 27019
 bindIp: cfg1.contoh.com

processManagement:
 timeZoneInfo: /usr/share/zoneinfo

security:
 keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
 replSetName: configReplSet

sharding:
 clusterRole: configsvr

server cfg2.contoh.com

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27019
  bindIp: cfg2.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
  replSetName: configReplSet

sharding:
  clusterRole: configsvr

server cfg3.contoh.com

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27019
  bindIp: cfg3.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
  replSetName: configReplSet

sharding:
  clusterRole: configsvr

Kemudian lakukan juga di semua server cfg1, cfg2, cfg3

sudo systemctl start mongod.service
sudo systemctl enable mongod.service

Lalu cek status dan pastikan ke 3 server statusnya running

sudo systemctl status mongod.service

Setup Config Server Replica

Config hanya di server cfg1.contoh.com, connect ke shell mongodb dengan perintah berikut

mongo --host cfg1.contoh.com --port 27019 -u administrator --authenticationDatabase admin

Akan diminta password admin yang sudah dibuat sebelumnya diatas

Menghubungkan semua server cfg1,cfg2,cfg3 dengan perintah berikut pada shell mongodb

rs.initiate({ _id: "configReplSet", configsvr: true, members: [{ _id : 0, host : "cfg1.contoh.com:27019"},{ _id : 1, host : "cfg2.contoh.com:27019"},{ _id : 2, host : "cfg3.contoh.com:27019"}]})

Untuk cek status

rs.status()

Berikut contoh status yang sudah connect semua server cfg

{
	"set" : "configReplSet",
	"date" : ISODate("2022-01-22T23:19:01.283Z"),
	"myState" : 1,
	"term" : NumberLong(1),
	"syncSourceHost" : "",
	"syncSourceId" : -1,
	"configsvr" : true,
	"heartbeatIntervalMillis" : NumberLong(2000),
	"majorityVoteCount" : 2,
	"writeMajorityCount" : 2,
	"votingMembersCount" : 3,
	"writableVotingMembersCount" : 3,
	"optimes" : {
		"lastCommittedOpTime" : {
			"ts" : Timestamp(1642893540, 1),
			"t" : NumberLong(1)
		},
		"lastCommittedWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
		"readConcernMajorityOpTime" : {
			"ts" : Timestamp(1642893540, 1),
			"t" : NumberLong(1)
		},
		"appliedOpTime" : {
			"ts" : Timestamp(1642893540, 1),
			"t" : NumberLong(1)
		},
		"durableOpTime" : {
			"ts" : Timestamp(1642893540, 1),
			"t" : NumberLong(1)
		},
		"lastAppliedWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
		"lastDurableWallTime" : ISODate("2022-01-22T23:19:00.303Z")
	},
	"lastStableRecoveryTimestamp" : Timestamp(1642893508, 1),
	"electionCandidateMetrics" : {
		"lastElectionReason" : "electionTimeout",
		"lastElectionDate" : ISODate("2022-01-22T13:57:17.441Z"),
		"electionTerm" : NumberLong(1),
		"lastCommittedOpTimeAtElection" : {
			"ts" : Timestamp(1642859827, 1),
			"t" : NumberLong(-1)
		},
		"lastSeenOpTimeAtElection" : {
			"ts" : Timestamp(1642859827, 1),
			"t" : NumberLong(-1)
		},
		"numVotesNeeded" : 2,
		"priorityAtElection" : 1,
		"electionTimeoutMillis" : NumberLong(10000),
		"numCatchUpOps" : NumberLong(0),
		"newTermStartDate" : ISODate("2022-01-22T13:57:17.478Z"),
		"wMajorityWriteAvailabilityDate" : ISODate("2022-01-22T13:57:18.531Z")
	},
	"members" : [
		{
			"_id" : 0,
			"name" : "cfg1.contoh.com:27019",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 34202,
			"optime" : {
				"ts" : Timestamp(1642893540, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2022-01-22T23:19:00Z"),
			"lastAppliedWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"lastDurableWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "",
			"electionTime" : Timestamp(1642859837, 1),
			"electionDate" : ISODate("2022-01-22T13:57:17Z"),
			"configVersion" : 1,
			"configTerm" : 1,
			"self" : true,
			"lastHeartbeatMessage" : ""
		},
		{
			"_id" : 1,
			"name" : "cfg2.contoh.com:27019",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 33714,
			"optime" : {
				"ts" : Timestamp(1642893539, 1),
				"t" : NumberLong(1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1642893539, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2022-01-22T23:18:59Z"),
			"optimeDurableDate" : ISODate("2022-01-22T23:18:59Z"),
			"lastAppliedWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"lastDurableWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"lastHeartbeat" : ISODate("2022-01-22T23:19:00.302Z"),
			"lastHeartbeatRecv" : ISODate("2022-01-22T23:19:00.013Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncSourceHost" : "cfg1.contoh.com:27019",
			"syncSourceId" : 0,
			"infoMessage" : "",
			"configVersion" : 1,
			"configTerm" : 1
		},
		{
			"_id" : 2,
			"name" : "cfg3.contoh.com:27019",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 33714,
			"optime" : {
				"ts" : Timestamp(1642893539, 1),
				"t" : NumberLong(1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1642893539, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2022-01-22T23:18:59Z"),
			"optimeDurableDate" : ISODate("2022-01-22T23:18:59Z"),
			"lastAppliedWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"lastDurableWallTime" : ISODate("2022-01-22T23:19:00.303Z"),
			"lastHeartbeat" : ISODate("2022-01-22T23:19:00.223Z"),
			"lastHeartbeatRecv" : ISODate("2022-01-22T23:19:00.256Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncSourceHost" : "cfg1.contoh.com:27019",
			"syncSourceId" : 0,
			"infoMessage" : "",
			"configVersion" : 1,
			"configTerm" : 1
		}
	],
	"ok" : 1,
	"$gleStats" : {
		"lastOpTime" : Timestamp(0, 0),
		"electionId" : ObjectId("7fffffff0000000000000001")
	},
	"lastCommittedOpTime" : Timestamp(1642893540, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1642893540, 1),
		"signature" : {
			"hash" : BinData(0,"TungAYubiqHZfoEyC7jMlStH1ZY="),
			"keyId" : NumberLong("7056029271826890775")
		}
	},
	"operationTime" : Timestamp(1642893540, 1)
}

quit()

Setting Shard Server

Mulai dari shrd1.contoh.com dan shrd2, shrd3 (rs0). Login server shrd1 edit /etc/mongod.conf

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27018
  bindIp: shrd1.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
  replSetName: rs0

sharding:
  clusterRole: shardsvr

Server shrd2.contoh.com

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27018
  bindIp: shrd2.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
  replSetName: rs0

sharding:
  clusterRole: shardsvr

Server shrd3.contoh.com

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27018
  bindIp: shrd3.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

replication:
  replSetName: rs0

sharding:
  clusterRole: shardsvr

Start MongoDB

sudo systemctl start mongod.service
sudo systemctl status mongod.service

Setup Shared Server

Login ke server shrd1.contoh.com, dan masuk ke shell mongo. Lakukan di hanya diserver shrd1

mongo --host shrd1.contoh.com --port 27018 -u administrator --authenticationDatabase admin

Masukkan password yang sudah dibuat diawal tutorial ini

Hubunkan semua server dari shell mongo dengan perintah

rs.initiate({ _id : "rs0", members:[{ _id : 0, host : "shrd1.contoh.com:27018" },{ _id : 1, host : "shrd2.contoh.com:27018" },{ _id : 2, host : "shrd3.contoh.com:27018" }]})

Bila muncul { “ok” : 1 } berarti shared server rs0 sudah berhasil connect semua.

quit()

Setting Query Router

Login server qrt1.contoh.com. Setting ini dilakukan juga di server qrt2.contoh.com

Stop mongodb

sudo systemctl stop mongod.service
sudo systemctl status mongod.service

Edit /etc/mongod.conf dan sesuaikan dengan config berikut

#storage:
#  dbPath: /var/lib/mongodb
#  journal:
#    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27017
  bindIp: qrt1.contoh.com

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /var/lib/mongodb/mongodb_key.pem

sharding:
  configDB: configReplSet/cfg1.contoh.com:27019,cfg2.contoh.com:27019,cfg3.contoh.com:27019

Kemudian edit /lib/systemd/system/mongod.service

sudo nano /lib/systemd/system/mongod.service

Samakan dengan config berikut

[Unit]
Description=Mongo Cluster Router
After=network.target

[Service]
User=mongodb
Group=mongodb

ExecStart=/usr/bin/mongos --config /etc/mongod.conf

LimitFSIZE=infinity
LimitCPU=infinity
LimitAS=infinity

LimitNOFILE=64000
LimitNPROC=64000

TasksMax=infinity
TasksAccounting=false

[Install]
WantedBy=multi-user.target

Start MongoDB

sudo systemctl start mongos.service
sudo systemctl status mongos.service

Menambahkan Shard ke Cluster

Login server qrt1.contoh.com, masuk shell mongo. Lakukan di satu server saja

mongo --host qrt1.contoh.com --port 27017 -u administrator --authenticationDatabase admin

Pada server qrt1 (rs0)

sh.addShard( "rs0/shrd1.contoh.com:27018,shrd2.contoh.com:27018,shrd2.contoh.com:27018")

Test Sharding

Membuat database pada mongodb, connect shell mongo

mongo --host qrt1.contoh.com --port 27017 -u administrator --authenticationDatabase admin

create database

use testDBdb.testCollection.ensureIndex( { _id : "hashed" } )

Mengaktifkan sharding untuk database baru

sh.enableSharding("testDB")
sh.shardCollection( "testDB.testCollection", { "_id" : "hashed" } )

kemudian kita lihat apakah testDB sudah terdistribusi

use configdb.databases.find()

Berikut hasilnya

{ "_id" : "testDB", "primary" : "rs0", "partitioned" : true, "version" : { "uuid" : UUID("97ea33eb-1f75-4783-8452-874ca0df5d56"), "timestamp" : Timestamp(1642872253, 1), "lastMod" : 1 } }

Mohon koreksi bila ada yang salah atau terlewat

Referensi

https://www.techsupportpk.com/2020/03/how-to-deploy-fault-tolerant-mongodb-sharded-cluster-ubuntu-debian.html

Tags: cluster mongodbsharded mongodb