This project is aimed at practicing Micro-service architecture. Design documents can be viewed here, make sure to open them with draw.io, as they contain the design flow for this project.
Refactoring from the previous Monolithic Simple Twitter API repo , live demo.
Before：
After：
The choice of databases is based on service characteristics, including:
- mysql
- mongodb
- redis: Records the subscriber list that behavior triggers need to notify.
For future message function expansion, the frontend can use Firebase's real-time database, but regarding room issues, a redesign is still necessary.
Because services are separated, there are data issues to address. Data synchronization utilizes asynchronous methods: Event Bus for handling. RabbitMQ is employed here. To manage concurrency issues, a version field is added to the data.
In terms of event publishing, hooks are mainly utilized.
To facilitate the management of different containers, Kubernetes is adopted. Detailed configuration files can be found in infra/k8s. CI/CD workflows will be implemented in the future.
(User authentication logic packaged into an npm package for use by each service)
Frontend:
- Users can create tweets, reply to tweets, and like tweets.
- Users can follow/subscribe (similar to FB notifications) to other users.
- Users can edit their profile (uploading background and profile pictures not yet implemented, hoping for future optimization to upload to specialized cloud storage services; originally relied on open-source services, as seen in the old project).
- Notification system.
Notify Server：
Backend: (Not implemented, done in the previous version)
- Administrators can delete tweets.
- Administrators can see all tweets.
- Administrators can see all user data.
Download the project to your local machine:
git clone https://github.com/JHIH-LEI/microservices-twitter-api-2022.git
Open Terminal and navigate to the folder where this project is stored:
cd micro-twitter-api-2022
First, find secret.yaml.example and database-secret.yaml.example in infra/k8s. Remove the "example" from the filename and fill in the missing values (encrypted using base64).
You can obtain the base64 value using the following command:
echo -n putValueHere | base64
cd ~
cd /etc
code hosts
Add:
127.0.0.1 twitter.dev
After saving, a prompt will appear in the lower-right corner:
Navigate to the root directory and run:
skaffold dev
Once you see the services listed as "listening on 3000" in the terminal, it means it's successful:
Now, you can access our backend API service using twitter.dev!
You can test it using Postman:
We will use an .env file to set up the environment variables needed for each service. Please rename the .env.example file to .env under the src folder of the corresponding service and fill in the values.
Start the server:
npm run dev
If you see the following message in the terminal, it means the server has been successfully established:
Example app listening on port 3000!
Run tests:
npm run test:ci
Sometimes, simply restarting may resolve the issue, but it could also be due to a failed image pull. While Skaffold automates sync, build, test, and deploy, it requires predefined images to start the automation process.
In this case, you can try building your own image and pushing it to Docker Hub for use. Remember to update the image name in the corresponding Kubernetes deployment.
Detailed Instructions:
First, navigate to each server to build the Docker image:
cd tweets
docker build -t yourDocerId/imageName .
Push to Docker Hub
docker push yourDocerId/imageName
Update the image location in the corresponding Kubernetes deployment:
Once everything is set up, return to the root directory and run:
skaffold dev
If you see each server successfully listening on port 3000 in the terminal without any error messages, everything is working fine.
此專案用於練習Micro-service。設計文件可至此觀看，記得用draw.io開啟，裡面包含了這次的設計流程。
重構之前的Monolithic Simple Twitter API repo 可點此，live demo點此，
Before：
After：
資料庫的選擇根據服務特性而選，分別採用了
- mysql
- mongodb
- redis：紀錄行為觸發者要通知的訂閱戶名單
未來若擴充訊息功能可以讓前端使用firebase的real time database，但針對房間的問題還需要重新設計。
因為是將服務分離，會有資料的問題需要處理，資料的同步利用了async的方法：Event Bus來處理，這邊採用了RabbitMQ來實作，為了處理concurrency的問題，在資料中加上version欄位來管理。
（使用者驗證的邏輯打包成npm package讓每個服務都能用） 前台：
- 使用者能新增推文、回覆推文、喜歡推文
- 使用者能追蹤/訂閱（類似於FB的通知）其他使用者
- 使用者能編輯個人資料（上傳背景及大頭照尚未實踐，未來希望是在優化方面能夠上傳到專門存放檔案的雲端服務，原本是仰賴開源服務，可見舊專案）
- 通知系統
Notify Server：
後台：(未實踐，前一個版本有做)
- 管理員能刪除推文
- 管理員能看見所有推文
- 管理員能看見所有使用者數據
下載專案到本地
git clone https://github.com/JHIH-LEI/microservices-twitter-api-2022.git
開啟終端機(Terminal)，進入存放此專案的資料夾
cd micro-twitter-api-2022
首先，請在infra/k8s 找到secret.yaml.example以及database-secret.yaml.example，把example結尾去除，並將缺少的值補齊（需使用base64加密）。
base64的值可以利用以下指令拿到：
echo -n putValueHere | base64
cd ~
cd /etc
code hosts
加上 127.0.0.1 twitter.dev
存檔後，右下會跳出：
到root dir運行：
skaffold dev
接著在終端機看到服務打印出listing on 3000就代表成功囉：
可以利用postman來做測試：
我們會利用.env file設定服務所需要的環境變數，請先在對應的服務其src資料夾底下，將.env.example改成.env，並填補值。
啟動伺服器
npm run dev
在終端機看到以下字串代表伺服器建立成功：
Example app listening on port 3000!
執行測試：
npm run test:ci
補充資料： 如果在使用skaffold dev 時遇到錯誤怎麼辦？
有時候只要重啟就沒事了，但也有可能是因為pull image失敗，skaffold雖然會幫我們自動sync, build, test, deploy，但在開始自動化流程之前，我們必須先有既定的image讓他能夠開始。
此時你可以嘗試自己建好image並推至docker hub來使用，請記得去對應的k8s depl把image的名稱改成你的。
詳細步驟指示：
首先，進入每一個server去建立docker image
cd tweets
docker build -t yourDocerId/imageName .
推到Docker Hub
docker push yourDocerId/imageName
至相對應的k8s depl 修改image位置
全部弄好了之後，回到root directory執行：
skaffold dev
在終端機有看到每個伺服器都成功監聽3000 port並沒有錯誤訊息就沒問題囉。