Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New LLM feature: Starred events to forensic report (stored as a Story) #3332

Open
wants to merge 84 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
b856f54
add response_schema support to ollama.py
itsmvd Feb 25, 2025
752ea57
Merge branch 'master' of github.com:itsmvd/timesketch
itsmvd Feb 25, 2025
5debf0f
Create separate llm provider directory, add response_schema to ollama…
itsmvd Feb 25, 2025
70d0699
Update timesketch.conf
itsmvd Feb 25, 2025
59ce086
solve naming conflict
itsmvd Feb 25, 2025
9e2c294
fix typo
itsmvd Feb 25, 2025
5f252a9
Add an __init__ file to the timsketch/lib/llms folder
itsmvd Feb 25, 2025
c040159
lint fix ollama
itsmvd Feb 25, 2025
7e4bed6
Merge branch 'google:master' into master
itsmvd Feb 26, 2025
9ab391e
Improve fallback mechanism for LLM configs
itsmvd Feb 26, 2025
5d4746a
formatting
itsmvd Feb 26, 2025
390cd09
format fix 2
itsmvd Feb 26, 2025
ad4d70b
Add LLM features manager and interface
itsmvd Feb 26, 2025
71fcdd9
Merge branch 'google:master' into master
itsmvd Feb 26, 2025
aa267cc
linter fix
itsmvd Feb 26, 2025
bd8d6d2
Automatically load features, add better doc-strings to interface.py
itsmvd Feb 27, 2025
290afc6
linter fix
itsmvd Feb 27, 2025
fb9b668
linter fixes
itsmvd Feb 27, 2025
0858b7f
linter fixes
itsmvd Feb 27, 2025
89c86d3
Merge branch 'google:master' into master
itsmvd Feb 27, 2025
1bcd2b1
Introduce LLMResource API method, tests, and add it as a method for t…
itsmvd Feb 27, 2025
be335ae
Merge branch 'google:master' into master
itsmvd Feb 27, 2025
f379b0e
linter fix
itsmvd Feb 27, 2025
2e669d0
linter fix
itsmvd Feb 27, 2025
1e58a28
Address comments from review
itsmvd Feb 28, 2025
3e21195
Merge branch 'google:master' into master
itsmvd Feb 28, 2025
37481e2
Add nl2q and llm_summarize as LLM features
itsmvd Feb 28, 2025
d028f0f
Couple of linter fixes on llm_summarize
itsmvd Feb 28, 2025
f4471b2
pylint: disable=protected-access
itsmvd Feb 28, 2025
e7e82d4
black formatting
itsmvd Feb 28, 2025
e171e4d
# pylint: disable=protected-access
itsmvd Feb 28, 2025
1cf49c0
formatting on nl2q
itsmvd Feb 28, 2025
0b3f251
add feature specific metrics
itsmvd Mar 1, 2025
bb2bc88
remove unnecessary comments
itsmvd Mar 1, 2025
5bf333e
fix black linter
jkppr Mar 3, 2025
d55b7b7
Merge branch 'master' into llm_shizzle
itsmvd Mar 4, 2025
9ebbb94
Merge branch 'google:master' into master
itsmvd Mar 4, 2025
6814079
review fixes
itsmvd Mar 4, 2025
25e7042
Handle incorrect prompt file + test
itsmvd Mar 4, 2025
222e9af
Merge branch 'google:master' into master
itsmvd Mar 4, 2025
e76ffcf
frontend: LLM features switch to new llm endpoint
itsmvd Mar 4, 2025
d997d3f
Merge branch 'google:master' into llm_shizzle
itsmvd Mar 4, 2025
a4276ee
layout fix
itsmvd Mar 4, 2025
8895e2a
Remove nl2q & llm_summarize features from the API
itsmvd Mar 4, 2025
766330e
Merge branch 'google:master' into master
itsmvd Mar 4, 2025
7cac889
Merge branch 'master' into llm_shizzle
itsmvd Mar 4, 2025
092711f
remove unused import
itsmvd Mar 4, 2025
9922596
Merge remote-tracking branch 'origin/llm_shizzle' into llm_shizzle
itsmvd Mar 4, 2025
ef1219c
Merge branch 'master' into llm_shizzle
jkppr Mar 5, 2025
1bd252c
Merge branch 'google:master' into master
itsmvd Mar 5, 2025
4f37d7f
Update RestApiClient in frontend-v3
itsmvd Mar 5, 2025
0abd597
Merge branch 'google:master' into llm_shizzle
itsmvd Mar 5, 2025
5df8c2e
Merge branch 'google:master' into master
itsmvd Mar 7, 2025
a1e7a29
Merge branch 'google:master' into master
itsmvd Mar 10, 2025
e2c3c66
Merge branch 'google:master' into llm_shizzle
itsmvd Mar 10, 2025
6549bbd
stash local changes
itsmvd Mar 10, 2025
9d8e343
Add nl2q and llm_summarize as LLM features
itsmvd Feb 28, 2025
1a213a2
Couple of linter fixes on llm_summarize
itsmvd Feb 28, 2025
3f9c7f0
pylint: disable=protected-access
itsmvd Feb 28, 2025
2baf0af
black formatting
itsmvd Feb 28, 2025
e46be97
# pylint: disable=protected-access
itsmvd Feb 28, 2025
be0b2ef
formatting on nl2q
itsmvd Feb 28, 2025
56bcd6d
add feature specific metrics
itsmvd Mar 1, 2025
09bca8b
remove unnecessary comments
itsmvd Mar 1, 2025
5478dcd
review fixes
itsmvd Mar 4, 2025
4f9a0b7
Handle incorrect prompt file + test
itsmvd Mar 4, 2025
a82d776
frontend: LLM features switch to new llm endpoint
itsmvd Mar 4, 2025
75f55f1
layout fix
itsmvd Mar 4, 2025
317eec8
Remove nl2q & llm_summarize features from the API
itsmvd Mar 4, 2025
3086dee
Merge branch 'llm_shizzle' of github.com:itsmvd/timesketch into llm_s…
itsmvd Mar 10, 2025
2bb1b68
Merge branch 'llm_shizzle' of github.com:itsmvd/timesketch into llm_s…
itsmvd Mar 10, 2025
a2f675d
Merge branch 'llm_shizzle' of github.com:itsmvd/timesketch into llm_s…
itsmvd Mar 10, 2025
bc20bf0
Make timeout configurable for snackBar methods
itsmvd Mar 11, 2025
27592c9
Re-applying changes, excluding snackbar timeout
itsmvd Mar 11, 2025
e794a0e
Merge branch 'llm_shizzle' of github.com:itsmvd/timesketch into llm_s…
itsmvd Mar 11, 2025
83bdf56
Merge branch 'google:master' into llm_shizzle
itsmvd Mar 11, 2025
024e862
UI changes for llm_forensic_report LLM feature
itsmvd Mar 11, 2025
ed1d5f5
update llm_forensic_reprot
itsmvd Mar 11, 2025
41e3512
Merge branch 'llm_shizzle' of github.com:itsmvd/timesketch into llm_s…
itsmvd Mar 11, 2025
e75f12d
latest changes to actions.py & llm_forensic_report
itsmvd Mar 11, 2025
7928776
fix debug totalHits
itsmvd Mar 11, 2025
3ffb54b
linter fix
itsmvd Mar 11, 2025
f3065b9
update prompt related settings
itsmvd Mar 11, 2025
26701bf
conf fix
itsmvd Mar 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions data/llm_forensic_report/prompt.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
You are a highly skilled digital forensic analyst. Your task is to analyze a set of security events, which have been identified as potentially significant ("starred events") in a Timesketch investigation. Based on these events, generate a concise forensic report summary, formatted in Markdown.

Focus on identifying:

* **Incident Overview:** Provide a brief summary of what appears to have happened based on these events. What type of incident is suggested (e.g., unauthorized access, malware infection, data breach attempt)?
* **Key Findings:** Highlight the most important observations and indicators from the events. Be specific and mention key entities (usernames, IP addresses, file paths, process names) involved.
* **Timeline of Significant Events (Chronological Order):** Briefly outline the sequence of key actions observed in the starred events.
* **Potential Impact/Severity:** Assess the potential impact or severity of the incident based on the available information.
* **Recommended Next Steps:** Suggest 2-3 concrete next steps for the investigation based on your analysis.

Use bolding (**...**) for key entities and findings. Format the output as a Markdown document.

Here are the events in JSON format: <events><EVENTS_JSON></events>
13 changes: 11 additions & 2 deletions data/timesketch.conf
Original file line number Diff line number Diff line change
Expand Up @@ -380,11 +380,17 @@ LLM_PROVIDER_CONFIGS = {
},
},
'llm_summarize': {
'aistudio': {
'model': 'gemini-2.0-flash-exp',
'vertexai': {
'model': 'gemini-2.0-flash-001',
'project_id': '',
},
},
'llm_forensic_report': {
'aistudio': {
'model': 'gemini-2.0-flash-001',
'api_key': '',
},
},
'default': {
'ollama': {
'server_url': 'http://ollama:11434',
Expand All @@ -401,3 +407,6 @@ EXAMPLES_NL2Q = '/etc/timesketch/nl2q/examples_nl2q'

# LLM event summarization configuration
PROMPT_LLM_SUMMARIZATION = '/etc/timesketch/llm_summarize/prompt.txt'

# LLM starred events to forensic report configuration
PROMPT_LLM_FORENSIC_REPORT = '/etc/timesketch/llm_forensic_report/prompt.txt'
1 change: 1 addition & 0 deletions docker/dev/build/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ if [ "$1" = 'timesketch' ]; then
ln -s /usr/local/src/timesketch/data/plaso_formatters.yaml /etc/timesketch/plaso_formatters.yaml
ln -s /usr/local/src/timesketch/data/nl2q /etc/timesketch/
ln -s /usr/local/src/timesketch/data/llm_summarize /etc/timesketch/
ln -s /usr/local/src/timesketch/data/llm_forensic_report /etc/timesketch/

# Set SECRET_KEY in /etc/timesketch/timesketch.conf if it isn't already set
if grep -q "SECRET_KEY = '<KEY_GOES_HERE>'" /etc/timesketch/timesketch.conf; then
Expand Down
12 changes: 12 additions & 0 deletions timesketch/frontend-ng/src/assets/main.scss
Original file line number Diff line number Diff line change
Expand Up @@ -208,3 +208,15 @@ html {
-o-transition: none !important;
transition: none !important;
}

$llm-gradient: linear-gradient(90deg,
#8ab4f8 0%,
#81c995 20%,
#f8c665 40%,
#ec7764 60%,
#b39ddb 80%,
#8ab4f8 100%);

:root {
--llm-gradient: #{$llm-gradient};
}
118 changes: 78 additions & 40 deletions timesketch/frontend-ng/src/components/Explore/EventList.vue
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,17 @@ limitations under the License.
<v-icon title="Download current view as CSV">mdi-download</v-icon>
</v-btn>

<v-btn
icon
@click="generateForensicReport()"
class="ml-2"
:loading="isGeneratingReport"
v-if="isStarredEventsFilterActive">
<div class="ts-llm-icon-wrapper" v-if="!isGeneratingReport">
<v-icon title="Generate forensic report with LLM from starred events">mdi-file-document-check</v-icon>
</div>
</v-btn>

<v-menu v-if="!disableSettings" offset-y :close-on-content-click="false">
<template v-slot:activator="{ on, attrs }">
<v-btn icon v-bind="attrs" v-on="on">
Expand Down Expand Up @@ -583,6 +594,7 @@ export default {
itemsPerPage: this.itemsPerPage,
},
isSummaryLoading: false,
isGeneratingReport: false,
currentItemsPerPage: this.itemsPerPage,
expandedRows: [],
selectedFields: [{ field: 'message', type: 'text' }],
Expand Down Expand Up @@ -621,6 +633,11 @@ export default {
}
},
computed: {
isStarredEventsFilterActive() {
return this.filterChips.some(chip =>
chip.type === 'label' && chip.value === '__ts_star'
)
},
summaryInfoMessage() {
const totalEvents = this.eventList.meta.summary_event_count
const uniqueEvents = this.eventList.meta.summary_unique_event_count
Expand Down Expand Up @@ -948,7 +965,6 @@ export default {
} else {
this.errorSnackBar(msg)
}
console.error('Error message: ' + msg)
console.error(e)
})
},
Expand All @@ -970,6 +986,35 @@ export default {
this.isSummaryLoading = false
})
},
generateForensicReport() {
if (this.totalHits > 1000) {
this.warningSnackBar('This feature is currently limited to a 1000 starred events, try setting a timerange filter. ' +
'This limit will be increased soon.', 10000);
return;
}

this.isGeneratingReport = true;
const requestData = {
filter: this.currentQueryFilter
};

ApiClient.llmRequest(this.sketch.id, 'llm_forensic_report', requestData)
.then((response) => {
this.isGeneratingReport = false;
if (response.data && response.data.story_id) {
this.$store.dispatch('updateSketch', this.sketch.id);
this.successSnackBar('Forensic report generated! You can find it in the "Stories" section.');
} else {
this.errorSnackBar('Error generating report. No story was created.');
}
})
.catch((error) => {
this.isGeneratingReport = false;
const errorMessage = (error.response && error.response.data && error.response.data.message) || 'Unknown error occurred';
this.errorSnackBar(`Error generating report: ${errorMessage}`);
console.error('Error generating starred events report:', error);
});
},
exportSearchResult: function () {
this.exportDialog = true
const now = new Date()
Expand Down Expand Up @@ -1247,20 +1292,19 @@ th:first-child {
padding: 0 0 0 10px !important;
}

.ts-event-list-container {
display: flex;
flex-direction: column;
width: 100%;
gap: 20px;
}
.ts-ai-summary-card {
border: 1px solid transparent !important;
border-radius: 8px;
background-color: #fafafa;
background-image:
linear-gradient(white, white),
linear-gradient(90deg,
#8ab4f8 0%,
#81c995 20%,
#f8c665 40%,
#ec7764 60%,
#b39ddb 80%,
#8ab4f8 100%
);
var(--llm-gradient);
background-origin: border-box;
background-clip: content-box, border-box;
background-size: 300% 100%;
Expand All @@ -1269,11 +1313,9 @@ th:first-child {
display: block;
margin-bottom: 20px;
}

.v-data-table {
display: block; /* Ensure block display for data table */
}

@keyframes borderBeamIridescent-subtle {
0% {
background-position: 0% 50%;
Expand All @@ -1282,25 +1324,16 @@ th:first-child {
background-position: 100% 50%;
}
}

.theme--dark.ts-ai-summary-card {
background-color: #1e1e1e;
border-color: hsla(0,0%,100%,.12) !important;
background-image:
linear-gradient(#1e1e1e, #1e1e1e),
linear-gradient(90deg,
#8ab4f8 0%,
#81c995 20%,
#f8c665 40%,
#ec7764 60%,
#b39ddb 80%,
#8ab4f8 100%
);
box-shadow: 0 2px 5px rgba(255, 255, 255, 0.08);
var(--llm-gradient);;
box-shadow: 0 2px 5px rgba(255, 255, 255, 0.08);
display: block;
margin-bottom: 20px;
}

.ts-ai-summary-text {
white-space: pre-line;
word-wrap: break-word;
Expand All @@ -1309,37 +1342,30 @@ th:first-child {
padding-left: 10px;
padding-right: 10px;
}

.ts-ai-summary-card .v-btn--icon {
cursor: pointer;
}

.ts-ai-summary-card .v-btn--icon:hover {
opacity: 0.8;
}

.ts-summary-placeholder-line {
height: 1em;
background-color: #e0e0e0;
margin-bottom: 0.5em;
border-radius: 4px;
width: 100%;
}

.ts-summary-placeholder-line.short {
width: 60%;
}

.ts-summary-placeholder-line.long {
width: 80%;
}

.shimmer {
background: linear-gradient(to right, #e0e0e0 8%, #f0f0f0 18%, #e0e0e0 33%);
background-size: 800px 100%;
animation: shimmer-animation 1.5s infinite linear forwards;
}

@keyframes shimmer-animation {
0% {
background-position: -468px 0;
Expand All @@ -1348,32 +1374,44 @@ th:first-child {
background-position: 468px 0;
}
}

.ts-event-list-container {
display: flex;
flex-direction: column;
width: 100%;
gap: 20px;
}

::v-deep .no-transition {
transition: none !important;
}

.ts-ai-summary-card-title {
display: flex;
align-items: baseline;
}

.ts-ai-summary-title {
margin-right: 8px;
font-weight: normal;
}

.ts-ai-summary-subtitle {
font-size: 0.7em;
color: grey;
vertical-align: middle;
display: inline-block;
}
.ts-llm-icon-wrapper {
position: relative;
display: inline-flex;
align-items: center;
justify-content: center;
}
.ts-llm-icon-wrapper::after {
content: "";
position: absolute;
top: -4px;
left: -4px;
right: -4px;
bottom: -4px;
border-radius: 50%;
background: var(--llm-gradient);
background-size: 300% 100%;
opacity: 0.2;
animation: borderBeamIridescent-subtle 6s linear infinite;
z-index: -1;
}
.v-btn:hover .ts-llm-icon-wrapper::after {
opacity: 0.4;
}
</style>
15 changes: 9 additions & 6 deletions timesketch/frontend-ng/src/mixins/snackBar.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,32 +23,35 @@ const defaultSnackBar = {
"timeout": defaultTimeout
}

// These methids will be available to all components without any further imports.
// These methods will be available to all components without any further imports.
Vue.mixin({
methods: {
successSnackBar(message) {
successSnackBar(message, timeout) {
let snackbar = defaultSnackBar
snackbar.message = message
snackbar.color = "success"
snackbar.timeout = timeout || defaultTimeout
this.$store.dispatch('setSnackBar', snackbar)
},
errorSnackBar(message) {
errorSnackBar(message, timeout) {
let snackbar = defaultSnackBar
snackbar.message = message
snackbar.color = "error"
snackbar.timeout = timeout || defaultTimeout
this.$store.dispatch('setSnackBar', snackbar)
},
warningSnackBar(message) {
warningSnackBar(message, timeout) {
let snackbar = defaultSnackBar
snackbar.message = message
snackbar.color = "warning"
snackbar.timeout = timeout || defaultTimeout
this.$store.dispatch('setSnackBar', snackbar)
},
infoSnackBar(message) {
infoSnackBar(message, timeout) {
let snackbar = defaultSnackBar
snackbar.message = message
snackbar.color = "info"
snackbar.timeout = 2000
snackbar.timeout = timeout || defaultTimeout
this.$store.dispatch('setSnackBar', snackbar)
},
}
Expand Down
Loading