Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore: 배포 자동화 및 Slack Webhook 설정 #29

Merged
merged 21 commits into from
Nov 1, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f92c827
Merge pull request #59 from kakao-tech-campus-2nd-step3/Develop
hynseoj Oct 11, 2024
62c3b31
Feat: POST 저장 기능, OCR 구현, 폴더 수정, 페이지별 조회 기능 구현 및 배포 (#66) (#67)
yugyeom-ghim Oct 25, 2024
e3e1c8d
Feat: AI서버에서 STT 결과 응답 왔을때 STT 결과 처리하는 기능 구현
yugyeom-ghim Oct 30, 2024
f823cb0
Merge branch 'Weekly9' into feature/#64-processing-stt-results
yugyeom-ghim Oct 30, 2024
2be29d4
Chore: 배포 환경 구성 및 빌드 설정 추가
yugyeom-ghim Oct 31, 2024
41b02fe
Refactor: tesseratct 관련 path를 설정 파일에서 관리하도록 변경
yugyeom-ghim Oct 31, 2024
b7e71c0
Chore: 서브모듈 추가
yugyeom-ghim Oct 31, 2024
6b457f5
Chore: .gitmodules 추가
yugyeom-ghim Oct 31, 2024
ed9920c
Chore: 서브모듈 제거
yugyeom-ghim Oct 31, 2024
8ac37cf
Chore: 서브모듈 추가
yugyeom-ghim Oct 31, 2024
4c84365
Chore: .gitmodules에 브랜치 명시
yugyeom-ghim Oct 31, 2024
61b2767
Chore: 서브모듈 URL을 SSH 방식으로 변경 (private 저장소 인증)
yugyeom-ghim Oct 31, 2024
f83955d
Chore: 토큰명 변경
yugyeom-ghim Oct 31, 2024
90415a0
Chore: working-directory 상대경로로 변경
yugyeom-ghim Oct 31, 2024
94371e1
Chore: 빌드 위치 디버깅 및 빌드 후 파일을 복사하는 방식으로 변경
yugyeom-ghim Oct 31, 2024
a0bebcf
Chore: profile을 prod로 변경
yugyeom-ghim Oct 31, 2024
4dca83b
Chore: 빌드파일만 복사하도록 변경
yugyeom-ghim Oct 31, 2024
79c531e
Feat: 잘못된 요청 또는 예상치 못한 에러를 보여주기 위한 슬렉 웹 훅 추가
yugyeom-ghim Nov 1, 2024
a261ef5
Chore: 스웨거 relesase 버전 0.0.2로 변경
yugyeom-ghim Nov 1, 2024
33a4792
Fix: OCR page넘버 i + 1로 계산 되는 부분 i로 변경
yugyeom-ghim Nov 1, 2024
cd02b28
Merge branch 'Weekly9' into release/0.0.2
yugyeom-ghim Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Deploy to Production

on:
push:
branches:
- 'release/**'
workflow_dispatch:

jobs:
deploy:
name: Production Deploy
runs-on: self-hosted
steps:
- name: Checkout Repository
uses: actions/checkout@v3
with:
submodules: true
token: ${{ secrets.ACTION_TOKEN }}

- name: Set up JDK 21
uses: actions/setup-java@v3
with:
java-version: '21'
distribution: 'corretto'

- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-

- name: Build with Gradle
run: |
pwd
ls -la
chmod +x ./gradlew
./gradlew clean bootJar

- name: Copy files and Deploy
run: |
cp -r ./build/libs/*.jar /home/yugyeom/notai/
cd /home/yugyeom/notai
docker-compose down
docker-compose up -d --build
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "Team29_BE_Submodule"]
path = Team29_BE_Submodule
url = git@github.com:29ana-notai/Team29_BE_Submodule.git
branch = release/0.0.2
5 changes: 0 additions & 5 deletions Dockerfile

This file was deleted.

1 change: 1 addition & 0 deletions Team29_BE_Submodule
Submodule Team29_BE_Submodule added at 9786d7
13 changes: 12 additions & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ plugins {
}

group = 'notai'
version = '0.0.1-SNAPSHOT'
version = '0.0.2'

java {
toolchain {
Expand Down Expand Up @@ -59,6 +59,9 @@ dependencies {

// OCR
implementation 'net.sourceforge.tess4j:tess4j:5.13.0'

// Slack
implementation("com.slack.api:slack-api-client:1.44.1")
}

tasks.named('test') {
Expand All @@ -70,3 +73,11 @@ test {
excludeTags 'exclude-test'
}
}

processResources.dependsOn('copySecret')

tasks.register('copySecret', Copy) {
from './Team29_BE_Submodule'
include "**"
into './src/main/resources'
}
2 changes: 1 addition & 1 deletion settings.gradle
Original file line number Diff line number Diff line change
@@ -1 +1 @@
rootProject.name = 'backend'
rootProject.name = 'notai'
10 changes: 5 additions & 5 deletions src/main/java/notai/client/ai/AiClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@

import notai.client.ai.request.LlmTaskRequest;
import notai.client.ai.response.TaskResponse;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.service.annotation.PostExchange;

import java.io.InputStream;

public interface AiClient {

@PostExchange(url = "/api/ai/llm")
TaskResponse submitLlmTask(@RequestBody LlmTaskRequest request);

@PostExchange(url = "/api/ai/stt")
TaskResponse submitSttTask(@RequestPart("audio") MultipartFile audioFile);
@PostExchange(url = "/api/ai/stt", contentType = MediaType.MULTIPART_FORM_DATA_VALUE)
TaskResponse submitSttTask(@RequestBody InputStream audioFileStream);
}

44 changes: 44 additions & 0 deletions src/main/java/notai/client/slack/SlackWebHookClient.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package notai.client.slack;

import com.slack.api.Slack;
import com.slack.api.webhook.Payload;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import notai.common.exception.ErrorMessages;
import notai.common.exception.type.ExternalApiException;
import org.springframework.http.HttpStatus;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Component;

import java.io.IOException;

@Component
@RequiredArgsConstructor
@Async
@Slf4j
public class SlackWebHookClient {

private final SlackWebHookProperty slackWebHookProperty;

public void sendToInfoChannel(String message) {
Slack slack = Slack.getInstance();
Payload payload = Payload.builder().text(message).build();

try {
slack.send(slackWebHookProperty.infoChannel().webhookUrl(), payload);
} catch (IOException e) {
throw new ExternalApiException(ErrorMessages.SLACK_API_ERROR, HttpStatus.INTERNAL_SERVER_ERROR.value());
}
}

public void sendToErrorChannel(String message) {
Slack slack = Slack.getInstance();
Payload payload = Payload.builder().text(message).build();

try {
slack.send(slackWebHookProperty.errorChannel().webhookUrl(), payload);
} catch (IOException e) {
throw new ExternalApiException(ErrorMessages.SLACK_API_ERROR, HttpStatus.INTERNAL_SERVER_ERROR.value());
}
}
}
14 changes: 14 additions & 0 deletions src/main/java/notai/client/slack/SlackWebHookProperty.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package notai.client.slack;

import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "slack")
public record SlackWebHookProperty(
WebhookProperty infoChannel,
WebhookProperty errorChannel
) {
public record WebhookProperty(
String webhookUrl
) {
}
}
2 changes: 1 addition & 1 deletion src/main/java/notai/common/config/SwaggerConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,6 @@ public OpenAPI openAPI() {
}

private Info apiInfo() {
return new Info().title("notai API").description("notai API 문서입니다.").version("0.0.1");
return new Info().title("notai API").description("notai API 문서입니다.").version("0.0.2");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,9 @@ public ApplicationException(ErrorMessages message, int code) {
super(message.getMessage());
this.code = code;
}

public ApplicationException(String message, int code) {
super(message);
this.code = code;
}
}
32 changes: 23 additions & 9 deletions src/main/java/notai/common/exception/ErrorMessages.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,19 @@ public enum ErrorMessages {
ANNOTATION_NOT_FOUND("주석을 찾을 수 없습니다."),

// document
DOCUMENT_NOT_FOUND("자료를 찾을 수 없습니다."), INVALID_DOCUMENT_PAGE("존재하지 않는 페이지 입니다."),
DOCUMENT_NOT_FOUND("자료를 찾을 수 없습니다."),
INVALID_DOCUMENT_PAGE("존재하지 않는 페이지 입니다."),

// ocr
OCR_RESULT_NOT_FOUND("OCR 데이터를 찾을 수 없습니다."), OCR_TASK_ERROR("PDF 파일을 통해 OCR 작업을 수행하는데 실패했습니다."),
OCR_RESULT_NOT_FOUND("OCR 데이터를 찾을 수 없습니다."),
OCR_TASK_ERROR("PDF 파일을 통해 OCR 작업을 수행하는데 실패했습니다."),

// folder
FOLDER_NOT_FOUND("폴더를 찾을 수 없습니다."),

// llm task
LLM_TASK_LOG_NOT_FOUND("AI 작업 기록을 찾을 수 없습니다."), LLM_TASK_RESULT_ERROR("AI 요약 및 문제 생성 중에 문제가 발생했습니다."),
LLM_TASK_LOG_NOT_FOUND("AI 작업 기록을 찾을 수 없습니다."),
LLM_TASK_RESULT_ERROR("AI 요약 및 문제 생성 중에 문제가 발생했습니다."),

// problem
PROBLEM_NOT_FOUND("문제 정보를 찾을 수 없습니다."),
Expand All @@ -33,19 +36,30 @@ public enum ErrorMessages {
RECORDING_NOT_FOUND("녹음 파일을 찾을 수 없습니다."),

// external api call
KAKAO_API_ERROR("카카오 API 호출에 예외가 발생했습니다."), AI_SERVER_ERROR("AI 서버 API 호출에 예외가 발생했습니다."),
KAKAO_API_ERROR("카카오 API 호출에 예외가 발생했습니다."),
AI_SERVER_ERROR("AI 서버 API 호출에 예외가 발생했습니다."),
SLACK_API_ERROR("슬랙 API 호출에 예외가 발생했습니다."),

// auth
INVALID_ACCESS_TOKEN("유효하지 않은 토큰입니다."), INVALID_REFRESH_TOKEN("유요하지 않은 Refresh Token입니다."), EXPIRED_REFRESH_TOKEN(
"만료된 Refresh Token입니다."), INVALID_LOGIN_TYPE("지원하지 않는 소셜 로그인 타입입니다."), NOTFOUND_ACCESS_TOKEN(
"토큰 정보가 존재하지 않습니다."),
INVALID_ACCESS_TOKEN("유효하지 않은 토큰입니다."),
INVALID_REFRESH_TOKEN("유요하지 않은 Refresh Token입니다."),
EXPIRED_REFRESH_TOKEN("만료된 Refresh Token입니다."),
INVALID_LOGIN_TYPE("지원하지 않는 소셜 로그인 타입입니다."),
NOTFOUND_ACCESS_TOKEN("토큰 정보가 존재하지 않습니다."),

// stt
STT_TASK_NOT_FOUND("음성 인식 작업을 찾을 수 없습니다."),
STT_TASK_ERROR("음성 인식 작업 중에 오류가 발생했습니다."),

// json conversion
JSON_CONVERSION_ERROR("JSON-객체 변환 중에 오류가 발생했습니다."),

// etc
INVALID_FILE_TYPE("지원하지 않는 파일 형식입니다."), FILE_NOT_FOUND("존재하지 않는 파일입니다."), FILE_SAVE_ERROR(
"파일을 저장하는 과정에서 오류가 발생했습니다."), INVALID_AUDIO_ENCODING("오디오 파일이 잘못되었습니다.");
INVALID_FILE_TYPE("지원하지 않는 파일 형식입니다."),
FILE_NOT_FOUND("존재하지 않는 파일입니다."),
FILE_SAVE_ERROR("파일을 저장하는 과정에서 오류가 발생했습니다."),
INVALID_AUDIO_ENCODING("오디오 파일이 잘못되었습니다."),
FILE_READ_ERROR("파일을 읽는 과정에서 오류가 발생했습니다.");

private final String message;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import notai.client.slack.SlackWebHookClient;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatusCode;
import org.springframework.http.ResponseEntity;
Expand All @@ -25,11 +26,19 @@
@RestControllerAdvice
public class ExceptionControllerAdvice extends ResponseEntityExceptionHandler {

private final SlackWebHookClient slackWebHookClient;

@ExceptionHandler(ApplicationException.class)
ResponseEntity<ExceptionResponse> handleException(HttpServletRequest request, ApplicationException e) {

log.info("잘못된 요청이 들어왔습니다. uri: {} {}, 내용: {}", request.getMethod(), request.getRequestURI(), e.getMessage());

slackWebHookClient.sendToInfoChannel(
"잘못된 요청이 들어왔습니다.\n" +
"uri: " + request.getMethod() + " " + request.getRequestURI() + "\n" +
"내용: " + e.getMessage()
);

requestLogging(request);

return ResponseEntity.status(e.getCode()).body(new ExceptionResponse(e.getMessage()));
Expand All @@ -39,6 +48,12 @@ ResponseEntity<ExceptionResponse> handleException(HttpServletRequest request, Ap
ResponseEntity<ExceptionResponse> handleException(HttpServletRequest request, Exception e) {
log.error("예상하지 못한 예외가 발생했습니다. uri: {} {}, ", request.getMethod(), request.getRequestURI(), e);

slackWebHookClient.sendToErrorChannel(
"예상하지 못한 예외가 발생했습니다.\n" +
"uri: " + request.getMethod() + " " + request.getRequestURI() + "\n" +
"내용: " + e.getMessage()
);

requestLogging(request);
return ResponseEntity.internalServerError().body(new ExceptionResponse(e.getMessage()));
}
Expand All @@ -57,6 +72,12 @@ protected ResponseEntity<Object> handleExceptionInternal(
HttpServletRequest request = ((ServletWebRequest) webRequest).getRequest();
log.error("예외가 발생했습니다. uri: {} {}, ", request.getMethod(), request.getRequestURI(), e);

slackWebHookClient.sendToErrorChannel(
"예외가 발생했습니다.\n" +
"uri: " + request.getMethod() + " " + request.getRequestURI() + "\n" +
" 내용: " + e.getMessage()
);

requestLogging(request);
return ResponseEntity.status(statusCode).body(new ExceptionResponse(e.getMessage()));
}
Expand Down
20 changes: 13 additions & 7 deletions src/main/java/notai/ocr/application/OCRService.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

Expand All @@ -22,20 +23,25 @@ public class OCRService {

private final OCRRepository ocrRepository;

@Value("${tesseract.library.path}")
private String libraryPath;

@Value("${tesseract.data.path}")
private String dataPath;

@Value("${tesseract.language}")
private String language;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

윈도우, 맥북마다 다른 변수를 이렇게 관리할 수 있다는 생각을 못했었네요 저는,,!
상수 관리 측면에서 너무 좋은 것 같습니다


@Async
public void saveOCR(
Document document, File pdfFile
) {
try {
// System.setProperty("jna.library.path", "/usr/local/opt/tesseract/lib/");
System.setProperty("jna.library.path", "C:\\Program Files\\Tesseract-OCR");
System.setProperty("jna.library.path", libraryPath);

//window, mac -> brew install tesseract, tesseract-lang
Tesseract tesseract = new Tesseract();

// tesseract.setDatapath("/usr/local/share/tessdata");
tesseract.setDatapath("C:\\Program Files\\Tesseract-OCR\\tessdata");
tesseract.setLanguage("kor+eng");
tesseract.setDatapath(dataPath);
tesseract.setLanguage(language);

PDDocument pdDocument = Loader.loadPDF(pdfFile);
PDFRenderer pdfRenderer = new PDFRenderer(pdDocument);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
package notai.pageRecording.domain;

import notai.pageRecording.query.PageRecordingQueryRepository;
import notai.recording.domain.Recording;
import org.springframework.data.jpa.repository.JpaRepository;

public interface PageRecordingRepository extends JpaRepository<PageRecording, Long> {
import java.util.List;

public interface PageRecordingRepository extends
JpaRepository<PageRecording, Long>, PageRecordingQueryRepository {

List<PageRecording> findAllByRecording(Recording recording);
}
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
package notai.pageRecording.query;

import com.querydsl.jpa.impl.JPAQueryFactory;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Repository;
import notai.pageRecording.domain.PageRecording;

@Repository
@RequiredArgsConstructor
public class PageRecordingQueryRepository {
import java.util.List;

private final JPAQueryFactory queryFactory;
public interface PageRecordingQueryRepository {

List<PageRecording> findAllByRecordingIdOrderByStartTime(Long recordingId);
}
Loading
Loading